We have had our glassfish instance go down every two weeks for a while with a java.lang.OutOfMemoryError: PermGen space. I increased the PermGen space to 512MB and startet dumping memory usage with jstat -gc. After two weeks I came up with the following graph that shows how the PermGen space is steadily increasing (the units on the x-axis are minutes, y-axis are KB).
I tried googling around for some kind of profiling tool that could pinpoint the error and a thread here on SO mentioned jmap, which proved to be quite helpful. Out of the approximately 14000 lines dumped from jmap -permstats $PID, approximately 12500 contained groovy/lang/GroovyClassLoader$InnerLoader, pointing to some kind of memory leak from either our own Groovy code or Groovy itself. I have to point out that Groovy constitues less than 1% of the relevant codebase .
Example output below:
class_loader classes bytes parent_loader alive? type
<bootstrap> 3811 14830264 null live <internal>
0x00007f3aa7e19d20 20 164168 0x00007f3a9607f010 dead groovy/lang/GroovyClassLoader$InnerLoader@0x00007f3a7afb4120
0x00007f3aa7c850d0 20 164168 0x00007f3a9607f010 dead groovy/lang/GroovyClassLoader$InnerLoader@0x00007f3a7afb4120
0x00007f3aa5d15128 21 181072 0x00007f3a9607f010 dead groovy/lang/GroovyClassLoader$InnerLoader@0x00007f3a7afb4120
0x00007f3aad0b40e8 36 189816 0x00007f3a9d31fbf8 dead org/apache/jasper/servlet/JasperLoader@0x00007f3a7d0caf00
....
So how can I proceed to find out more about what code is causing this?
From this article I infer that our Groovy code is dynamically creating classes somewhere. And from the the dump from jmap I can see that most of