Why does Hadoop need classes like Text or IntWritable instead of String or Integer?

Question

Why does Hadoop need to introduce these new classes? They just seem to complicate the interface

Answer 1

Some more good info:

they’ve got two features that are relevant

they have the “Writable” interface -they know how to write to a DataOutput stream and read from a DataInput stream -explicitly.

they have their contents updates via the set() operation. This lets you reuse the same value, repeatedly, without creating new instances. It’s a lot more efficient if the same mapper or reducer is called repeatedly: you just create your instances of the writables in the constructor and reuse them

In comparison, Java’s Serializable framework “magically” serializes objects -but it does it in a way that is a bit brittle and is generally impossible to read in values generated by older versions of a class. the Java Object stream is designed to send a graph of objects back -it has to remember every object reference pushed out already, and do the same on the way back. The writables are designed to be self contained.

This is from: http://hortonworks.com/community/forums/topic/why-hadoop-uses-default-longwritable-or-intwritable/