How to emit 2D double array from mapper using TwoDArrayWritable

问题

I want to emit a 2D double array using TwoDArrayWritable as value .

how to write the context.write(key , )

EDIT

And in Reducer how to get them in a Two Dimensional double array and print the values.

I Wrote in Mapper

row = E.length;
col = E[0].length;
                     TwoDArrayWritable array = new TwoDArrayWritable (DoubleWritable.class);
                     DoubleWritable[][] myInnerArray = new DoubleWritable[row][col];
                     // set values in myInnerArray
                     for (int k1 = 0; k1 < row; k1++) {
                        for(int j1=0;j1< col;j1++){
                            myInnerArray[k1][j1] = new DoubleWritable(E[k1][j1]);

                    }
                 array.set(myInnerArray);
                 context.write(clusterNumber, array);

But showing a Nullpointer exception

13/11/01 16:34:07 INFO mapred.LocalJobRunner: Map task executor complete.
13/11/01 16:34:07 WARN mapred.LocalJobRunner: job_local724758890_0001
java.lang.Exception: java.lang.NullPointerException
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:404)
Caused by: java.lang.NullPointerException
    at org.apache.hadoop.io.TwoDArrayWritable.write(TwoDArrayWritable.java:91)
    at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:100)
    at org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:84)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:945)
    at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:601)
    at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:85)
    at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:106)
    at edu.Mapper.map(Mapper.java:277)


Mapper.java:277 : context.write(clusterNumber, array);

回答1:

 TwoDArrayWritable array = new TwoDArrayWritable (DoubleWritable.class);
 DoubleWritable[][] myInnerArray = new DoubleWritable[10][];
 // set values in myInnerArray
 array.set(myInnerArray);
 context.write(key, array);

I believe the NPE is because some element of the array is null. From the code you posted I wonder does ecol equal col.

回答2:

Guys This NPE exception occured dew to some null value in your array. If any value not there in array than it will put null dew to object in nature. You can check this in debugging mode. So your task is to check there is no null value in your array.

来源：https://stackoverflow.com/questions/19721836/how-to-emit-2d-double-array-from-mapper-using-twodarraywritable

标签

Hadoop

MapReduce