Byte array of unknown length in java

Question

I am constructing an array of bytes in java and I don\'t know how long the array will be.

I want some tool like Java\'s StringBuffer that you can just call .append

Answer 1

I wrote one that is really easy to use and avoids a lot of byte array buffer copying.

It has one method called add.

You can add strings, bytes, byte, long, int, double, float, short, and chars to it.

The API is easy to use and somewhat fail safe. It does not allow you to copy the buffer around and does not promote having two readers.

It has a bounds check mode and a I KNOW WHAT I AM DOING MODE with no bounds checking.

The bounds check mode auto-grows it so there is no hassle.

https://github.com/RichardHightower/boon/wiki/Auto-Growable-Byte-Buffer-like-a-ByteBuilder

Here is a complete step by step guide on how to use it. It is on github.

Java Boon - Auto Growable Byte Buffer like a ByteBuilder

Have you ever wanted an easy to use buffer array that grow automatically and/or you can give it a fix size and just add stuff to it? I have. I wrote one too.

Look.. I can write strings to it (it converts them to UTF-8).

    ByteBuf buf = new ByteBuf();
    buf.add(bytes("0123456789\n"));
    buf.add("0123456789\n");
    buf.add("0123456789\n");
    buf.add("0123456789\n");
    buf.add("0123456789\n");
    buf.add("0123456END\n");

Then later I can read the String out of the buffer:

    String out = new String(buf.readAndReset(), 0, buf.len());
    assertEquals(66, buf.len());
    assertTrue(out.endsWith("END\n"));

I never have to set the size of the array. It will auto-grow as needed in an efficient manner.

If I know exactly how large my data is going to be than I can save some bounds checking by using createExact.

    ByteBuf buf = ByteBuf.createExact(66);
    buf.add(bytes("0123456789\n"));
    buf.add("0123456789\n");
    buf.add("0123456789\n");
    buf.add("0123456789\n");
    buf.add("0123456789\n");
    buf.add("0123456END\n");
    assertEquals(66, buf.len());

If I use create exact, then I am saying... hey.. I know exactly how big it can grow to and it will never go over this number and if it does...you can hit me over the head with a sack of rocks!

The following hits you over the head with a sack of rocks! THROWS AN EXCEPTION!!!!

    ByteBuf buf = ByteBuf.createExact(22);
    buf.add(bytes("0123456789\n"));
    buf.add("0123456789\n");
    buf.add("0123456789\n");
    buf.add("0123456789\n");
    buf.add("0123456789\n");
    buf.add("0123456END\n");

It works with doubles.

    ByteBuf buf = ByteBuf.createExact(8);

    //add the double
    buf.add(10.0000000000001);

    byte[] bytes = buf.readAndReset();
    boolean worked = true;

    worked |= idxDouble(bytes, 0) == 10.0000000000001 || die("Double worked");

It works with float.

    ByteBuf buf = ByteBuf.createExact(8);

    //add the float
    buf.add(10.001f);

    byte[] bytes = buf.readAndReset();
    boolean worked = true;

    worked |= buf.len() == 4 || die("Float worked");


    //read the float
    float flt = idxFloat(bytes, 0);

    worked |= flt == 10.001f || die("Float worked");

It works with int.

    ByteBuf buf = ByteBuf.createExact(8);

    //Add the int to the array
    buf.add(99);

    byte[] bytes = buf.readAndReset();
    boolean worked = true;


    //Read the int back
    int value = idxInt(bytes, 0);

    worked |= buf.len() == 4 || die("Int worked length = 4");
    worked |= value == 99 || die("Int worked value was 99");

It works with char.

    ByteBuf buf = ByteBuf.createExact(8);

    //Add the char to the array
    buf.add('c');

    byte[] bytes = buf.readAndReset();
    boolean worked = true;


    //Read the char back
    int value = idxChar(bytes, 0);

    worked |= buf.len() == 2 || die("char worked length = 4");
    worked |= value == 'c' || die("char worked value was 'c'");

It works with short.

    ByteBuf buf = ByteBuf.createExact(8);

    //Add the short to the array
    buf.add((short)77);

    byte[] bytes = buf.readAndReset();
    boolean worked = true;


    //Read the short back
    int value = idxShort(bytes, 0);

    worked |= buf.len() == 2 || die("short worked length = 2");
    worked |= value == 77 || die("short worked value was 77");

It even works with bytes.

    ByteBuf buf = ByteBuf.createExact(8);

    //Add the byte to the array
    buf.add( (byte)33 );

    byte[] bytes = buf.readAndReset();
    boolean worked = true;


    //Read the byte back
    int value = idx(bytes, 0);

    worked |= buf.len() == 1 || die("byte worked length = 1");
    worked |= value == 33 || die("byte worked value was 33");

You can add all sorts of primitives to your byte array.

    boolean worked = true;
    ByteBuf buf = ByteBuf.create(1);

    //Add the various to the array
    buf.add( (byte)  1 );
    buf.add( (short) 2 );
    buf.add( (char)  3 );
    buf.add(         4 );
    buf.add( (float) 5 );
    buf.add( (long)  6 );
    buf.add( (double)7 );

    worked |= buf.len() == 29 || die("length = 29");


    byte[] bytes = buf.readAndReset();

    byte myByte;
    short myShort;
    char myChar;
    int myInt;
    float myFloat;
    long myLong;
    double myDouble;

Now we just verify that we can read everything back.

    myByte    =   idx       ( bytes, 0 );
    myShort   =   idxShort  ( bytes, 1 );
    myChar    =   idxChar   ( bytes, 3 );
    myInt     =   idxInt    ( bytes, 5 );
    myFloat   =   idxFloat  ( bytes, 9 );
    myLong   =    idxLong   ( bytes, 13 );
    myDouble  =   idxDouble ( bytes, 21 );

    worked |= myByte   == 1 || die("value was 1");
    worked |= myShort  == 2 || die("value was 2");
    worked |= myChar   == 3 || die("value was 3");
    worked |= myInt    == 4 || die("value was 4");
    worked |= myFloat  == 5 || die("value was 5");
    worked |= myLong   == 6 || die("value was 6");
    worked |= myDouble == 7 || die("value was 7");

Once you call

 byte[] bytes = buf.readAndReset() 

then you are saying that you are done with the ByteBuffer!

Once you ask for the bytes, it becomes useless as it sets the internal byte array to nothing.

When you call readAndReset, it is giving you its buffer. Here is my internal state, you can have it, but I am going to set it to null so nobody else uses it.

It is ok. Just create another if you are sure only one instance at a time is using the buffer (byte []).

You can even use the buffer you were just using as in

ByteBuf buf2 = new ByteBuf.create(bytes); 

This is because no buffer gets copied. ByteBuf writes to the buffer you give it. If you want another copy to be given to ByteBuf then do this:

ByteBuf buf2 = new ByteBuf.create( copy(bytes) ); 

This is boon after all. :)

Come check out boon. You get the above class and idx, and idxInt and idxLong for free!

https://github.com/RichardHightower/boon/