Why does BinaryWriter prepend gibberish to the start of a stream? How do you avoid it?

Question

I\'m debugging some issues with writing pieces of an object to a file and I\'ve gotten down to the base case of just opening the file and writing \"TEST\" in it. I\'m doing

Answer 1

Remember that Java strings are internally encoded in UTF-16.

So, "test" is actually made of the bytes 0xff, 0xfe (together the byte order mark), 0x74, 0x00, 0x65, 0x00, 0x73, 0x00, 0x74, 0x00.

You probably want to work with bytes instead of streams of characters.

Answer 2

That's because a BinaryWriter is writing the binary representation of the string, including the length of the string. If you were to write straight data (e.g. byte[], etc.) it won't include that length.

byte[] text = System.Text.Encoding.Unicode.GetBytes("test");
FileStream fs = new FileStream("C:\\test.txt", FileMode.Create);
BinaryWriter writer = new BinaryWriter(fs);
writer.Write(text);
writer.Close();

You'll notice that it doesn't include the length. If you're going to be writing textual data using the binary writer, you'll need to convert it first.

Answer 3

The byte at the start is the length of the string, it's written out as a variable-length integer.

If the string is 127 characters or less, the length will be stored as one byte. When the string hits 128 characters, the length is written out as 2, and it will move to 3 and 4 at some lengths as well.

The problem here is that you're using BinaryWriter, which writes out data that BinaryReader can read back in later. If you wish to write out in a custom format of your own, you must either drop writing strings like that, or drop using BinaryWriter altogether.