Tips for keeping Perl memory usage low

Question

What are some good tips for keeping memory usage low in a Perl script? I am interested in learning how to keep my memory footprint as low as possible for systems depending o

Answer 1

In addition to brian d foy's suggestions, I found the following also helped a LOT.

1. Where possible, don't "use" external modules, you don't know how much memory they utilise. I found by replacing the LWP and HTTP::Request::Common modules with either Curl or Lynx slashed memory usage by half.
2. Slashed it again by modifying our own modules and pulling in only the required subroutines using "require" rather than a full library of unnecessary subs.

3. Brian mentions using lexical variables with the smallest possible scope. If you're forking, using "undef" also helps by immediately freeing up memory for Perl to re-use. So you declare a scalar, array, hash or even sub, and when you're finished with any of them, use :

   my (@divs) = localtime(time); $VAR{minute} = $divs[1];

   undef @divs; undef @array; undef $scalar; undef %hash; undef ⊂

4. And don't use any unnecssary variables to make your code smaller. It's better to hard code whatever is possible to reduce namespace usage.

Then there's a lot of other tricks you can try depending on your application's functionality. Ours was run by cron, every minute. We found we could fork half the processes with a sleep(30) so half would run and complete within the first 30 seconds, freeing up cpu and memory, and the other half would run after a 30 second delay. Halved the resource usage again. All up, we managed to reduce RAM usage from over 2 GB down to 200MB, a 90% saving.

We managed to get a pretty good idea of memory usage with

top -M

as our script was executed on an relatively stable server with only one site. So watching "free ram" gave us a pretty good indication of memery usage.

Also "ps" grepping for your script and if forking, sorting by either memory or cpu usage was a good help.

ps -e -o pid,pcpu,pmem,stime,etime,command --sort=+cpu | grep scriptname | grep -v grep