map[byte]byte{0:10} should be using least 2 bytes, one for value and one per key. But as each hashmap implmentation, there is also a hidden cost per item.
What is the memory
It seems like there is a buffer involved, and it grows only when needed. I can't tell for gccgo, though, I just tried it on the playground. Basically, it allocates 128 bytes for the empty map, then it grows when needed.
You can see it here : http://play.golang.org/p/RjohbSOq0x