Ruby leaks memory

I’ve spent a considerable amount of time with various tools attempting to figure out why it is that our thin processes (and mongrels before them) grow so egregiously. Typically they reach about 450Mb in a day, after which we restart them via monit.What makes them grow? Well, we are fetching a lot of stuff from the DB all the time – meaning that thousands of small strings are being instantiated – so perhaps we can attribute some growth to heap fragmentation. But we tried changing to ptmalloc3 – it didn’t help; in fact I think in our case this is rather a red herring.In an effort to get the problem under control, I wrote a plugin to reduce the number of strings that are made, changing the implementation of the mysql library so that all_hashes actually returns fake hashes that are implemented as arrays – to prevent all those column names being saved as frozen strings (for the hash keys) for every row that is fetched from the DB. But that didn’t help much, if at all, either.But whilst I was playing with ruby with valgrind, I noticed some memory going missing. At first I thought it was probably me. But with further investigation I found a simple expression that makes ruby leak.

a = eval "b=0"

It’s actually the eval that leaks – the a = is not really needed, but it makes the leak show as a definite leak as opposed to a possible one in this simple one liner. If you want to leak a lot of memory, this is the way:

def grow  for i in 1..100    eval "b#{i}=1"  endend15000.times {grow}

You can fiddle with the numbers to make it grow as much as you like.Valgrind reports the leak like this (this one made by running the loop 5000.times):

==18706== 217,988,864 bytes in 499,985 blocks aredefinitely lost in loss record 6 of 6==18706==    at 0x4A05AF7: realloc (vg_replace_malloc.c:306)==18706==    by 0x432398: ruby_xrealloc (gc.c:151)==18706==    by 0x465E9C: local_append (parse.y:5649)==18706==    by 0x465F64: local_cnt (parse.y:5667)==18706==    by 0x4646AC: assignable (parse.y:4902)==18706==    by 0x458E80: ruby_yyparse (parse.y:844)==18706==    by 0x45E5F4: yycompile (parse.y:2606)==18706==    by 0x45E8F4: rb_compile_string (parse.y:2676)==18706==    by 0x41DDF3: compile (eval.c:6412)==18706==    by 0x41E289: eval (eval.c:6493)==18706==    by 0x41E817: rb_f_eval (eval.c:6611)==18706==    by 0x41C765: call_cfunc (eval.c:5700)==18706==    by 0x41BB04: rb_call0 (eval.c:5856)==18706==    by 0x41D291: rb_call (eval.c:6103)==18706==    by 0x415182: rb_eval (eval.c:3494)

The memory is allocated when ruby is expanding its local variable table in the parser. But what I don’t know yet is exactly where to add a call to free to release that memory. I’m hoping that someone over at ruby-core can help. Interestingly, it appears that Rubinius leaks too, which is surprising given that it is a completely new implementation.I’m not the only one to have found a leak in Ruby lately – I wonder if the issues with god are related to this?Fixing this leak may not completely cure our Rails memory growth problem (probably won’t), but at least it will help.

Advertisements

6 thoughts on “Ruby leaks memory

  1. Interesting! Keep us posted. We are a lot more stable since we compiled against "ptmalloc3" – Which version of Ruby are you using?BestZeno

  2. This is Ruby 1.8.6 p114. We are certainly not seeing pauses due to GC running, so I suspect that we are not hitting quite the same problems that you had.

  3. I wonder if tcmalloc would help, too. [i.e. it’s also out there and seems to work well, perhaps without some of the drawbacks of ptmalloc3 + ruby]. It won’t work on OS X but on Linux it does.

  4. Yes, we use OS X for dev, and so far I have been unable to compile ruby 1.8.7 with tcmalloc on that platform.However, it does compile ok on our x86-64 app server, but I have yet to complete testing with it so I haven’t run it live so far. I hear that performance is significantly improved with tcmalloc, but I don’t expect memory usage improvements.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s