How to reduce memory usage in Ruby

Question

Sidekiq workers consume a lot of memory: how can I fix that?

I have a Rails application that can usually run with about 1 GB of RAM. However there are some Sidekiq workers that constantly grow over time - for days - and consume a lot of memory (e.g. 2.5GB).

Are there any solutions?

Answer

The problem is related to Ruby threads and how memory allocation works at low-level. In many cases it’s not a memory leak, but just a memory bloat.

Here’s two effective solution that we have used in production to reduce the memory usage:

• Reduce the number of Sidekiq threads (e.g. use 10 threads instead of 25)

• Set the environment variable MALLOC_ARENA_MAX=2 for tuning glibc memory behavior.

The environment variable is a no-brainer. Here’s the results that we got by adding it (max RAM in 16h):

• web: 1191Mi -> 966Mi (~20% less memory)

• worker type 1: 911Mi -> 348Mi (~60% less memory)

• worker type 2: 549Mi -> 304Mi (~45% less memory)

Note that:

• Heroku sets that variable automatically

• the creator of Sidekiq recommends it

• the maintainer of Puma recommends it