Grep across multiple files in Hadoop Filesystem

前端 未结 5 824
时光说笑
时光说笑 2020-12-30 02:01

I am working with Hadoop and I need to find which of ~100 files in my Hadoop filesystem contain a certain string.

I can see the files I wish to search like this:

5条回答
  •  没有蜡笔的小新
    2020-12-30 02:30

    You are looking to applying grep command on hdfs folder

    hdfs dfs -cat /user/coupons/input/201807160000/* | grep -c null
    

    here cat recursively goes through all files in the folder and I have applied grep to find count.

提交回复
热议问题