Casting a byte array to a managed structure

Question

Update: Answers to this question helped me code the open sourced project AlicanC\'s Modern Warfare 2 Tool on GitHub. You can see how I am reading these packets in MW

Answer 1

This is how i did:

using System;
using System.Runtime.InteropServices;
public static object GetObjectFromBytes(byte[] buffer, Type objType)
{
    object obj = null;
    if ((buffer != null) && (buffer.Length > 0))
    {
        IntPtr ptrObj = IntPtr.Zero;
        try
        {
            int objSize = Marshal.SizeOf(objType);
            if (objSize > 0)
            {
                if (buffer.Length < objSize)
                    throw new Exception(String.Format("Buffer smaller than needed for creation of object of type {0}", objType));
                ptrObj = Marshal.AllocHGlobal(objSize);
                if (ptrObj != IntPtr.Zero)
                {
                    Marshal.Copy(buffer, 0, ptrObj, objSize);
                    obj = Marshal.PtrToStructure(ptrObj, objType);
                }
                else
                    throw new Exception(String.Format("Couldn't allocate memory to create object of type {0}", objType));
            }
        }
        finally
        {
            if (ptrObj != IntPtr.Zero)
                Marshal.FreeHGlobal(ptrObj);
        }
    }
    return obj;
}

And in the struct definition I didn't use any fixed region, instead I used the MarshalAs attribute if the standard marshalling didn't worked. This is what you will probally need for the string.

You would use this function like this:

PacketHeader ph = (PacketHeader)GetObjectFromBytes(buffer, typeof(PacketHeader));

Edit: I didn´t see your BigEndian "restriction" in the code example. This solution will only work if the bytes are LittleEndian.

Edit 2: In the string of your example you would decorate it with:

[MarshalAs(UnmanagedType.LPStr)]

In the arrays I would go with something like this for a n-sized array:

[MarshalAs(UnmanagedType.ByValArray, SizeConst = n)]

Answer 2

Well, you have two tasks here really. First is to interpret byte[] as struct essentially and second is to deal with possible different endianness.

So, they are somewhat diverge. AFAIK if you want to use marshaling - it will just interpret bytes as if it were managed structure. So converting from one endian to another is left to you. It is not hard to do but it will not be automatic.

So, to interpret byte[] as struct you have to have something like that:

[StructLayout(LayoutKind.Sequential)]
internal struct X
{
    public int IntValue;
    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 3, ArraySubType = UnmanagedType.U1)] 
    public byte[] Array;
}

static void Main(string[] args)
{
    byte[] data = {1, 0, 0, 0, 9, 8, 7}; // IntValue = 1, Array = {9,8,7}
    IntPtr ptPoit = Marshal.AllocHGlobal(data.Length);
    Marshal.Copy(data, 0, ptPoit, data.Length);
    var x = (X) Marshal.PtrToStructure(ptPoit, typeof (X));
    Marshal.FreeHGlobal(ptPoit);

    Console.WriteLine("x.IntValue = {0}", x.IntValue);
    Console.WriteLine("x.Array = ({0}, {1}, {2})", x.Array[0], x.Array[1], x.Array[2]);
}

So first 4 bytes go to IntValue (1,0,0,0) -> [little endian] -> 1 Next 3 bytes go directly to array.

If you want BigEndian you should do it yourself:

int LittleToBigEndian(int littleEndian)
{
    byte[] buf = BitConverter.GetBytes(littleEndian).Reverse().ToArray();
    return BitConverter.ToInt32(buf, 0);
}

It is somewhat messy like that, so probably for you will be better to stick with your custom-written parser that takes bytes one-by-one from source byte[] and fill your data class without StructLayout and other native interop.

Answer 3

//I have found this at: http://code.cheesydesign.com/?p=572 (I have not tested yet, but // at first sight it will work well.)

    /// <summary>
    /// Reads in a block from a file and converts it to the struct
    /// type specified by the template parameter
    /// </summary>
    /// <typeparam name="T"></typeparam>
    /// <param name="reader"></param>
    /// <returns></returns>
    private static T FromBinaryReader<T>(BinaryReader reader)
    {

        // Read in a byte array
        byte[] bytes = reader.ReadBytes(Marshal.SizeOf(typeof(T)));

        // Pin the managed memory while, copy it out the data, then unpin it
        GCHandle handle = GCHandle.Alloc(bytes, GCHandleType.Pinned);
        T theStructure = (T)Marshal.PtrToStructure(handle.AddrOfPinnedObject(), typeof(T));
        handle.Free();

        return theStructure;
    }

Answer 4

To convert a byte array to a string you do this;

byte [] dBytes = ...
string str;
System.Text.UTF8Encoding enc = new System.Text.UTF8Encoding();
str = enc.GetString(dBytes);

And to convert the string back to a byte array

public static byte[] StrToByteArray(string str)
{
    System.Text.UTF8Encoding  encoding=new System.Text.UTF8Encoding();
    return encoding.GetBytes(str);
}

Now read your string and see what your data is.

Answer 5

I'd turn the byte array into a memory stream. Then instantiate a binary reader on that stream. And then define helper functions that take a binary reader and parse a single class.

The built in BinaryReader class always uses little endian.

I'd use classes instead of structs here.

class PacketHeader 
{
    uint16_t magic;
    uint16_t packet_size;
    uint32_t unknown1;
    uint32_t unknown2;
    uint32_t unknown3;
    uint32_t unknown4;
    uint16_t unknown5;
    uint16_t unknown6;
    uint32_t unknown7;
    uint32_t unknown8;
    string packet_type; // replaced with a real string
};

PacketHeader ReadPacketHeader(BinaryReader reader)
{
  var result=new PacketHeader();
  result.magic = reader.ReadInt16();
  ...
  result.packet_type=ReadCString();//Some helper function you might need to define yourself
  return result;
}

Answer 6

My approach is different. I did not want to copy any byte.
I just wanted to use them, modify some of them and use changed byte[] array in other place as byte[].
After digging google and stackoverflow I decided to go into unsafe/fixed.
There playing with code I have found fast code without copy.
This is DEBUG/TEST code. Check this in debug mode.
Remeber that this way you do not make a copy and you are working on raw byte[] data.
Any change in struct will reflect in byte[] array change and vice versa.
++TESTED++ WORKS

//FOR DEBUG/TEST ONLY
using System.Runtime.InteropServices;
namespace ByteStructCast1
{
    class Program
    {
        [StructLayout(LayoutKind.Sequential, Pack = 1)]
        unsafe struct StructTest//4B
        {
            [MarshalAs(UnmanagedType.U2)]
            public ushort item1;//2B
            public fixed byte item2[2];//2B =2x 1B
        }
        static void Main(string[] args)
        {
            //managed byte array
            byte[] DB1 = new byte[7];//7B more than we need. byte buffer usually is greater.
            DB1[0] = 2;//test data |> LITTLE ENDIAN
            DB1[1] = 0;//test data |
            DB1[2] = 3;//test data
            DB1[3] = 4;//test data
            unsafe //OK we are going to pin unmanaged struct to managed byte array
            {
                fixed(byte* db1 = DB1) //db1 is pinned pointer to DB1 byte[] array
                {
                    //StructTest t1 = *(StructTest*)db1;    //does not change DB1/db1
                    //t1.item1 = 11;                        //does not change DB1/db1
                    db1[0] = 22;                            //does CHANGE DB1/db1
                    DB1[0] = 33;                            //does CHANGE DB1/db1
                    StructTest* ptest = (StructTest*)db1;   //does CHANGE DB1/db1
                    ptest->item1 = 44;                      //does CHANGE DB1/db1
                    ptest->item2[0]++;                      //does CHANGE DB1/db1
                    ptest->item2[1]--;                      //does CHANGE DB1/db1
                }
            }
        }
    }
}