How to get the input file name in the mapper in a Hadoop program?

后端 未结 10 2060
粉色の甜心
粉色の甜心 2020-11-29 18:48

How I can get the name of the input file within a mapper? I have multiple input files stored in the input directory, each mapper may read a different file, and I need to kno

10条回答
  •  没有蜡笔的小新
    2020-11-29 19:31

    First you need to get the input split, using the newer mapreduce API it would be done as follows:

    context.getInputSplit();
    

    But in order to get the file path and the file name you will need to first typecast the result into FileSplit.

    So, in order to get the input file path you may do the following:

    Path filePath = ((FileSplit) context.getInputSplit()).getPath();
    String filePathString = ((FileSplit) context.getInputSplit()).getPath().toString();
    

    Similarly, to get the file name, you may just call upon getName(), like this:

    String fileName = ((FileSplit) context.getInputSplit()).getPath().getName();
    

提交回复
热议问题