Go conversion between struct and byte array

Question

I am writing a client - server application in Go. I want to perform C-like type casting in Go.

E.g. in Go

type packet struct {
    opcode uint16
    data [1024]byte
}

var pkt1 packet
...
n, raddr, err := conn.ReadFromUDP(pkt1)  // error here

Also I want to perform C-like memcpy(), which will allow me to directly map the network byte stream received to a struct.

e.g. with above received pkt1

type file_info struct {
    file_size uint32       // 4 bytes
    file_name [1020]byte
}

var file file_info
if (pkt1.opcode == WRITE) {
    memcpy(&file, pkt1.data, 1024)
}

Answer 1

unsafe.Pointer is, well, unsafe, and you don't actually need it here. Use encoding/binary package instead:

// Create a struct and write it.
t := T{A: 0xEEFFEEFF, B: 3.14}
buf := &bytes.Buffer{}
err := binary.Write(buf, binary.BigEndian, t)
if err != nil {
    panic(err)
}
fmt.Println(buf.Bytes())

// Read into an empty struct.
t = T{}
err = binary.Read(buf, binary.BigEndian, &t)
if err != nil {
    panic(err)
}
fmt.Printf("%x %f", t.A, t.B)

Playground

As you can see, it handles sizes and endianness quite neatly.

Answer 2

Thank you for answers and I am sure they work perfectly. But in my case I was more interested in parsing the []byte buffer received as network packet. I used following method to parse the buffer.

var data []byte // holds the network packet received
opcode := binary.BigEndian.Uint16(data) // this will get first 2 bytes to be interpreted as uint16 number
raw_data := data[2:len(data)] // this will copy rest of the raw data in to raw_data byte stream

While constructing a []byte stream from a struct, you can use following method

type packet struct {
    opcode uint16
    blk_no uint16
    data   string
}
pkt := packet{opcode: 2, blk_no: 1, data: "testing"}
var buf []byte = make([]byte, 50) // make sure the data string is less than 46 bytes
offset := 0
binary.BigEndian.PutUint16(buf[offset:], pkt.opcode)
offset = offset + 2
binary.BigEndian.PutUint16(buf[offset:], pkt.blk_no)
offset = offset + 2
bytes_copied := copy(buf[offset:], pkt.data)

I hope this gives general idea about how to convert []byte stream to struct and struct back to []byte stream.

Answer 3

You'd have to use unsafe, also uint is 8 bytes on 64bit systems, you have to use uint32 if you want 4 bytes.

It's ugly, unsafe and you have to handle endianess yourself.

type packet struct {
    opcode uint16
    data   [1022]byte
}

type file_info struct {
    file_size uint32     // 4 bytes
    file_name [1018]byte //this struct has to fit in packet.data
}

func makeData() []byte {
    fi := file_info{file_size: 1 << 20}
    copy(fi.file_name[:], []byte("test.x64"))
    p := packet{
        opcode: 1,
        data:   *(*[1022]byte)(unsafe.Pointer(&fi)),
    }
    mem := *(*[1022]byte)(unsafe.Pointer(&p))
    return mem[:]
}

func main() {
    data := makeData()
    fmt.Println(data)
    p := (*packet)(unsafe.Pointer(&data[0]))
    if p.opcode == 1 {
        fi := (*file_info)(unsafe.Pointer(&p.data[0]))
        fmt.Println(fi.file_size, string(fi.file_name[:8]))
    }
}

play

Answer 4

I've had the same problem and I solved it by using the "encoding/binary" package. Here's an example:

package main

import (
  "bytes"
  "fmt"
  "encoding/binary"
)

func main() {
  p := fmt.Println
  b := []byte{43, 1, 0}

  myStruct := MyStruct{}
  err := binary.Read(bytes.NewBuffer(b[:]), binary.BigEndian, &myStruct)

  if err != nil {
    panic(err)
  }

  p(myStruct)
}

type MyStruct struct {
  Num uint8
  Num2 uint16
}

Here's the running example: https://play.golang.org/p/Q3LjaAWDMh