From where does the program allocate memory?

Question

As a C and C++ programmer I have used malloc and new to allocate memory. I am just wondering: how does the OS allocate memory?

1. D

Answer 1

This answer is going to be pretty Linux-centric

It's a lot more complicated than it seems. When you allocate a block of memory, one of two things might happen. If your process already has enough memory that was previously freed but not returned to the OS (allocators usually don't), that memory will be marked as allocated in the allocators tables, and will be returned. When the process doesn't already have the memory, the allocator asks the OS for more. On Linux, that means the brk or sbrk syscalls. I don't know what it means on Windows or OSX.

All modern, common OSes (Linux, Windows, OSX, and their derivatives) use virtual memory, the concept of addresses not necessarily pointing at real RAM. Until you put something in it, the memory you allocated might not exist at all. Once you start using it, the OS needs to make room for it in RAM. To do this, the OS will store other, unused pages (parts of memory) in a swap file or partition on the disk, if it is so configured.

The OS is constantly moving pages of memory in and out between swap storage and actual RAM. Whenever your process accesses a part of memory that isn't in RAM right now, the processor generates a page fault, which causes the OS to load that page(s) from the disk, and make room in memory for them by moving other pages into swap.

---

So basically, in a modern OS, memory can come from anywhere the OS chooses, and might not even exist when you've allocated it.

---

Note: Some OSes, like Linux most of the time, will even let you allocate memory that it doesn't have to give you. This is called overcommiting memory.