How much memory is consumed by the Linux kernel per TCP/IP network connection?

Question

How much memory on average is consumed by the Linux kernel (in kernel address space) per TCP/IP network connection?

Answer 1

For a TCP connection memory consumed depends on

1. size of sk_buff (internal networking structure used by linux kernel)

2. the read and write buffer for a connection

the size of buffers can be tweaked as required

root@x:~# sysctl -A | grep net | grep mem

check for these variables

these specify the maximum default memory buffer usage for all network connections in kernel

net.core.wmem_max = 131071

net.core.rmem_max = 131071

net.core.wmem_default = 126976

net.core.rmem_default = 126976

these specify buffer memory usage specific to tcp connections

net.ipv4.tcp_mem = 378528   504704  757056

net.ipv4.tcp_wmem = 4096    16384   4194304

net.ipv4.tcp_rmem = 4096    87380   4194304

the three values specified are " min default max" buffer sizes. So to start with linux will use the default values of read and write buffer for each connection. As the number of connection increases , these buffers will be reduced [at most till the specified min value] Same is the case for max buffer value.

These values can be set using this sysctl -w KEY=KEY VALUE

eg. The below command ensures the read and write buffers for each connection are 4096 each.

sysctl -w net.ipv4.tcp_rmem='4096 4096 4096'

sysctl -w net.ipv4.tcp_wmem='4096 4096 4096'

Answer 2

Also depends on which layer. In case of a pure bridging scenario, there's just the bridge-level FDB. When routing comes into play, there's the routing table and the IP-level FDB/neighbor db. And finally, once a local socket is in the play, you have of course window size, socket buffers (both send and receive, and they default to 128k last time I checked), fragment lists (if used), so that is where your memory goes, but a clear-cut answer is hard to make with all the parts in use. You can use ss -m to obtain a few memory statistics of local stream sockets.

Answer 3

It depends. On many many things.
I think an idle connection will take a few hundreds of bytes.
But if there's data in the transmit and/or receive data, then the consumption increases. The window size can roughly limit this consumption.
The extra consumption for data isn't just the bytes in the receive/transmit queue. There are overheads, so a segment with one byte might take something like 2K. TCP tries to reduce this, for example by merging segments into a single sk_buff, but it doesn't always succeed.