Tips for keeping Perl memory usage low

Question

What are some good tips for keeping memory usage low in a Perl script? I am interested in learning how to keep my memory footprint as low as possible for systems depending o

Answer 1

My two dimes.

1. Do threads created in Perl prevent copying Perl module libraries into memory for each thread?

   • It does not, it is just one process, what isn't repeated in the program stack, each thread must have its own.

2. Is threads (use threads) the most efficient way (or the only) way to create threads in Perl?

   • IMO Any method eventually calls the pthread library APIs which actually does the work.

3. In threads, I can specify a stack_size paramater, what specifically should I consider when specifying this value, and how does it impact memory usage?

   • Since threads runs in the same process space, the stack cannot be shared. The stack size tells pthreads how far away they should be from each other. Everytime a function is called the local variables are allocated on the stack. So stack size limits how deep you can recurse. you can allocate as little as possible to the extend that your application still works.

With threads in Perl/Linux, what is the most reliable method to determine the actual memory usage on a per-thread basis?

* Stack storage is fixed after your thread is spawned, heap and static storage is shared and
  they can be used by any thread so this notion of memory usage per-thread doesn't really
  apply. It is per process.


Comparing fork and thread:

* fork duplicate the process and inherites the file handles

  advantages: simpler application logic, more fault tolerant.
              the spawn process can become faulty and leaking resource
              but it will not bring down the parent. good solution if
              you do not fork a lot and the forked process eventually
              exits and cleaned up by the system.

  disadvantages: more overhead per fork, system limitation on the number
              of processes you can fork. You program cannot share variables.

* threads runs in the same process with addtional program stacks.

  advantages: lower memory footprint, thread spawn if faster and ligther
              than fork. You can share variables.

  disadvantages: more complex application logic, serialization of resources etc.
              need to have very reliable code and need to pay attention to
              resource leaks which can bring down the entire application.

IMO, depends on what you do, fork can use way less memory over the life time of the 
application run if whatever you spawn just do the work independently and exit, instead of
risking memory leaks in threads.