Why does Node.js have incremental memory usage?

Question

I have a gameserver.js file that is well over 100 KB in size. And I kept checking my task manager after each refresh on my browser and kept seeing my node.exe m

Answer 1

The garbage collector is not called all the time because it blocks your process. So V8 launches GC when it thinks it's necessary.

To find if you have a memory leak I propose to fire up the GC manually after every request just to see if your memory is still going up. Normally if you don't have a memory leak your memory should not increase. Because the GC will clean all non-used objects. If your memory is still going up after a GC call you have a memory leak.

To launch GC manually you can do that, but attention! Don't use this in production; this is just a way to cleanup your memory and see if you have a memory leak.

Launch Node.js like this:

node --expose-gc --always-compact test.js

It will expose the garbage collector and force it to be aggressive. Call this method to run the GC:

global.gc();

Call this method after each hit on your server and see if the GC clean the memory or not.

You can also do two heapdumps of your process before and after request to see the difference.

Don't use this in production or in your project. It is just a way to see if you have a memory leak or not.

Answer 2

Seeing an increased memory footprint by a Node.js application is completely normal behaviour. Node.js constantly analyses your running code, generates optimised code, reverts to unoptimised code (if needed), etc. All this requires quite a lot of memory even for the most simple of applications (Node.js itself is from a large part written in JavaScript that follows the same optimisations/deoptimisations as your own code).

Additionally, a process may be granted more memory when it needs it, but many operating systems remove that allocated memory from the process only when they decide it is needed elsewhere (i.e. by another process). So an application can, in peaks, consume 1 GB of RAM, then garbage collection kicks in, usage drops to 500 MB, but the process may still keep the 1 GB.

Detecting presence of memory leaks

To properly analyse memory usage and memory leaks, you must use Node.js's process.memoryUsage().

You should set up an interval that dumps this memory usage into a file i.e. every second, then apply some "stress" on your application over several seconds (i.e. for web servers, issue several thousand requests). Then take a look at the results and see if the memory just keeps increasing or if it follows a steady pattern of increasing/decreasing.

Detecting source of memory leaks

The best tool for this is likely node-heapdump. You use it with the Chrome debugger.

1. Start your application and apply initial stress (this is to generate optimised code and "warm-up" your application)
2. While the app is idle, generate a heapdump
3. Perform a single, additional operation (i.e. one more request) that you suspect will likely cause a memory leak - this is probably the trickiest part especially for large apps
4. Generate another heapdump
5. Load both heapdumps into Chrome debugger and compare them - if there is a memory leak, you will see that there are some objects that were allocated during that single request but were not released afterwards
6. Inspect the object to determine where the leak occurs

I had the opportunity to investigate a reported memory leak in the Sails.js framework - you can see detailed description of the analysis (including pretty graphs, etc.) on this issue.

There is also a detailed article about working with heapdumps by StrongLoop - I suggest to have a look at it.