PermGen problems with Lift and Jetty

Question

I'm developing on the standard Lift platform (maven and jetty). I'm repeatedly (once every couple of days) getting this:

Exception in thread "7048009@qtp-3179125-12" java.lang.OutOfMemoryError: PermGen space
2009-09-15 19:41:38.629::WARN:  handle failed
java.lang.OutOfMemoryError: PermGen space

This is in my dev environment. It's not a problem because I can keep restarting the server. In deployment I'm not having these problems so it's not a real issue. I'm just curious.

I don't know too much about the JVM. I think I'm correct in thinking that permanent generation memory is for things like classes and interned strings? What I remember is a bit mixed up with the .NET memory model...

Any reason why this is happening? Are the defaults just crazily low? Is it to do with all the auxiliary objects that Scala has to create for Function objects and similar FP things? Every time I restart Jetty with newly written code (every few minutes) I imagine it re-loads classes etc. But even so, it cant' be that many can it? And shouldn't the JVM be able to deal with a large number of classes?

Cheers

Joe

Answer 1

From this post:

This exception occurred for one simple reason :
the permgenspace is where class properties, such as methods, fields, annotations, and also static variables, etc. are stored in the Java VM, but this space has the particularity to not being cleaned by the garbage collector. So if your webapp uses or creates a lot of classes (I’m thinking dynamic generations of classes), chances are you met this problem. Here are some solutions that helped me get rid of this exception :

• -XX:+CMSClassUnloadingEnabled : this setting enables garbage collection in the permgenspace
• -XX:+CMSPermGenSweepingEnabled : allows the garbage collector to remove even classes from the memory
• -XX:PermSize=64M -XX:MaxPermSize=128M : raises the amount of memory allocated to the permgenspace

May be this could help.

Edit July 2012 (almost 3 years later):

Ondra Žižka comments (and I have updated the answer above):

JVM 1.6.0_27 says: Please use:

• CMSClassUnloadingEnabled (Whether class unloading enabled when using CMS GC)
• in place of CMSPermGenSweepingEnabled in the future

See the full Hotspot JVM Options - The complete reference for mroe.

Answer 2

If you see this when running mvn jetty:run, set the MAVEN_OPTS.

For Linux:

export MAVEN_OPTS="-XX:+CMSClassUnloadingEnabled -XX:PermSize=256M -XX:MaxPermSize=512M"
mvn jetty:run

For Windows:

set "MAVEN_OPTS=-XX:+CMSClassUnloadingEnabled -XX:PermSize=256M -XX:MaxPermSize=512M"
mvn jetty:run

Should be fine now. If not, increase -XX:MaxPermSize.

You can also put these permanently to your environment.

• For Linux, append the export line to ~/.bashrc

• For Windows, press Win-key + PrintScreen, and go Advanced > Environment. See also http://support.microsoft.com/kb/310519.

Answer 3

This is because of the reloading of classes as you suggested. If you are using lots of libraries etc. the sum of classes will rapidly grow for each restart. Try monitoring your jetty instance with VisualVM to get an overview of memory consumption when reloading.

Answer 4

The mailing list (http://groups.google.com/group/liftweb/) is the official support forum for Lift, and where you'll be able to get a better answer. I don't know the particulars of your dev setup (you don't go into much detail), but I assume you're reloading your war in Jetty without actually restarting it. Lift doesn't perform dynamic class generation (as suggested by VonC above), but Scala compiles each closure as a separate class. If you're adding and removing closures to your code over the course of several days, it's possible that too many classes are being loaded and never unloaded and taking up perm space. I'd suggest you enable the options JVM options mentioned by VonC above and see if they help.

Answer 5

The permanent generation is where the JVM puts stuff that will probably not be (garbage) collected like custom classloaders.

Depending on what you are deploying, the perm gen setting can be low. Some applications and/or containers combination do contain some memory leaks, so when an app gets undeployed sometimes some stuff like class loaders are not collected, resulting in filling the Perm Space thus generating the error you are having.

Unfortunately, currently the best option in this case is to max up the perm space with the following jvm flag (example for 192m perm size):

-XX:MaxPermSize=192M (or 256M)

The other option is to make sure that either the container or the framework do not leak memory.