Efficient Go serialization of struct to disk

Question

I\'ve been tasked to replace C++ code to Go and I\'m quite new to the Go APIs. I am using gob for encoding hundreds of key/value entries to disk pages but the gob encoding h

Answer 1

"Manual coding", you're so afraid of, is trivially done in Go using the standard encoding/binary package.

You appear to store string length values as 32-bit integers in big-endian format, so you can just go on and do just that in Go:

package main

import (
    "bytes"
    "encoding/binary"
    "fmt"
    "io"
)

func encode(w io.Writer, s string) (n int, err error) {
    var hdr [4]byte
    binary.BigEndian.PutUint32(hdr[:], uint32(len(s)))
    n, err = w.Write(hdr[:])
    if err != nil {
        return
    }
    n2, err := io.WriteString(w, s)
    n += n2
    return
}

func main() {
    var buf bytes.Buffer

    for _, s := range []string{
        "ab",
        "cd",
        "de",
    } {
        _, err := encode(&buf, s)
        if err != nil {
            panic(err)
        }
    }
    fmt.Printf("%v\n", buf.Bytes())
}

Playground link.

Note that in this example I'm writing to a byte buffer, but that's for demonstration purposes only—since encode() writes to an io.Writer, you can pass it an opened file, a network socket and anything else implementing that interface.