Gemfile of dreams: the libraries we use to build Rails apps

The Rails ecosystem is quite vast. For (pretty much) every typical task, there are a number of libraries waiting to help you. How can we pick the right one for the job? We could rely on some specific metrics, like GitHub stars or the number of downloads (see The Ruby Toolbox for valuable insights). Or, we could draw from past experience: this approach works well but could lead to extreme conservatism.

At Evil Martians, we bet on our collective experience and alien instinct when choosing a gem or deciding to build a new one. The benefit of this collective experience is that it spans dozens of minds and years: we know which wins to repeat and which fails to avoid. While we cannot turn this knowledge into a super-smart chatty AI (yet), we can share a glimpse of it in this post.

The rest of the article is structured to resemble a Rails application Gemfile, and it contains the following sections:

• Rails fundamentals
• Background jobs
• Active Record extensions
• Authentication and authorization
• HTML views
• Asset management
• Crafting APIs (and GraphQL)
• Logging and instrumentation
• Development tools
• Testing tools
• Everything else

Rails fundamentals

To begin, we need to configure the fundamental elements of a Rails web application: a web server and a database.
For our database, we’ll bet on PostgreSQL, while Puma will be our web server of choice. So, the beginning of our Gemfile looks like this:

gem 'rails', '~> 7.0.4'
gem 'pg'
gem 'puma'

There’s not much to add here, so let’s move on to the third pillar of Ruby on Rails applications: the background jobs engine.

Background jobs

The vital characteristic of web application performance is throughput (how many requests we can handle in a given time). Due to Ruby (MRI) concurrency limitations, more precisely Global Virtual Machine Lock (GVL), the number of requests that can be served in parallel is capped. To increase the throughput, we try hard to minimize request time; the best way to achieve this is by offloading as much work as possible to background execution. That’s where a background jobs processing engine comes into play.

Rails provides an abstraction, Active Job, to define background jobs. But it’s up to users to choose a specific processor implementation. No surprises here—we use Sidekiq:

gem 'sidekiq'

There are plenty of Sidekiq add-ons (including official Pro and Enterprise versions) which give you better control over background job execution logic. Let me share two of the most frequently used within Martian projects:

gem 'sidekiq-grouping'
gem 'sidekiq-limit_fetch'

The sidekiq-grouping gem allows you to buffer enqueued jobs and process them in batches. This is helpful when the enqueue rate is high, but you don’t need jobs to be instantly processed. A typical use case is re-indexing models, or broadcasting live updates to models.

To keep hard-working (and RAM-bloating) jobs under control, we use sidekiq-limit_fetch. For example, to avoid generating more than a single large XSLX export at a time, we configure a limit for the queue:

# sidekiq.yml
---
:queues:
  - default
  # ...
  - exports
:process_limits:
  exports: 1

To finish up this section, let me also mention some gems beyond Sidekiq:

gem 'schked'

For recurrent jobs, we use a lightweight solution called Schked. It’s a wrapper over the well-known rufus-scheduler with some useful additional features, such as testing support and Rails Engines awareness.

gem 'faktory_worker_ruby'

Finally, in multi-language projects, we use Faktory to enqueue and process jobs from different services. For example, an ML service written in Python can send analysis results to a Rails application, and vice versa: a Rails app can send analysis requests to a Python app.

Active Record extensions

As you can see from the section title, we do not “derail” from the Rails Way: we use Active Record to model business logic and communicate with a database.

Active Record is the most feature-rich Rails sub-framework, and its API is growing with every Rails release (we’ll even get Common Table Expressions support in Rails 7.1!) However, there’s still room for even more features, so we’ll add a bunch of plugins to make that happen.

Since we specialize in PostgreSQL, we use its specific features heavily. My personal favorite is JSONB. When used reasonably, it can give you a huge productivity boost (no need to worry about schema, migrations, and so on.) By default, Rails converts JSONB values to plain Ruby hashes, which are not so powerful in terms of modeling business logic. That’s why we usually back our unstructured fields using one of the following libraries:

gem 'store_attribute'
gem 'store_model'

The store_attribute gem extends the built-in Active Record store_accessor feature and adds type-casting support. Accordingly, your JSONB values can be treated as regular attributes.

Store Model goes further and allows you to define model classes backed by JSON attributes.

The other database features (not PostgreSQL, but SQL specific) from our Top-5 list are views, triggers, full-text search, and enum types. Here are the corresponding gems:

gem 'activerecord-postgres_enum'
gem 'pg_search'
gem 'postgresql_cursor'

gem 'fx'
gem 'scenic'
# or
gem 'pg_trunk'

By the way, PostgreSQL enums support is coming to Rails: (part 1 has been merged, part 2 is just waiting for its time to come).

Speaking of feature-like extensions, I’d like to mention just a couple of them:

# Soft-deletion
gem 'discard'
# Helpers to group by time periods
gem 'groupdate'

Discard provides a minimalistic soft-deletion functionality (with no default_scope attached). Being minimalistic makes it easy to customize to your needs, that’s why this gem is present in most applications where we softly delete data.

Groupdate is a must-have if you deal with time-aggregated reporting (and still haven’t migrated to a time-series database, like, say, TimescaleDB).

Authentication and authorization

Most of the applications I’ve worked on in the past five years have had this line in their Gemfile:

gem 'devise'

And I’d bet that nine out of ten readers have Devise in their bundles, too; this is just the current reality. (But, nevertheless, I want to dream.) In a less common situation, when building a Rails application from scratch, we always consider alternative authentication libraries first. And the leader is… drumroll… Rails itself! You can go pretty far with the has_secure_password feature.

And there are a couple more candidates before giving up and falling back to Devise:

gem 'sorcery'
gem 'jwt_sessions'

Sorcery has been around for years. It’s less magical (despite stating the opposite in its README) and opinionated than Devise, and has a pluggable architecture (so you can easily pick the parts you need).

The JWT Sessions library provides everything you need to build token-based authentication. It’s perfect for building API-only Rails apps with mobile or SPA frontends.

Let’s skip the whole category of OAuth-driven authentication solutions (this question deserves its own post) and switch to authorization.

Wait, what? Isn’t that the same thing? Not at all. The fundamental difference comes down to the questions we’re answering. Authentication answers “Who’s there?”, while authorization answers “Am I allowed to do that?“. Thus, authorization deals with roles and permissions.

Authorization itself could be split into two components: an authorization model and authorization (enforcement) layer. The former represents the business logic behind permissions (whether you use roles, granular permissions, or whatever). The latter one, the authorization layer, defines how to apply the authorization rules.

An authorization model is too application-specific, while for implementing an authorization layer, we can use generic techniques (wrapped into gems). Historically, we’ve used Pundit to implement authorization enforcement, but today we have a better option:

gem 'action_policy'

Action Policy is Pundit on steroids. It uses the same concepts (policies) but provides more out-of-the-box features (performance and is developer-experience oriented.)

HTML views

The classic Rails Way assumes an HTML-first approach. A server is responsible for each part of the Model-View-Controller paradigm: that is, the “M”, the “V”, and the “C”. After the dark ages of API-only Rails apps, we’ve recently witnessed a renaissance of the HTML-over-the-wire approach in the Rails community. So, here we are, crafting view templates again!

However, in the 2020s, the way to do that is different as the toolbox:

gem 'view_component'
gem 'view_component-contrib'
gem 'lookbook', require: false

gem 'turbo-rails'

Hotwire makes our HTML-based applications interactive and reactive, while view components help us organize the templates and their logic.

ViewComponent in the Wild I: building modern Rails frontends

ViewComponent in the Wild I: building modern Rails frontends

October 12, 2022

Read also

Asset management

Dealing with assets in Rails has become tricky since the introduction of Webpacker in Rails 5. Today, in Rails 7, we have multiple official solutions to the problem (Import Maps, JS/CSS bundling gems). On Mars, however, we go another way:

gem 'vite_rails'

Using Vite is a middle ground between backend-oriented and frontend-oriented assets management. It’s simple to use for both Hotwire applications and React SPAs (and we often have hybrid setups). Further, the vite_ruby gem provides a five-star developer experience.

And here are a couple more assets-related goodies to add to our file:

gem 'imgproxy'
gem 'premailer-rails'

Do you have to deal with a lot of user-generated content? Don’t waste your Ruby server resources for image transformations—let a dedicated proxy server do the hard work. Obviously, we use our own imgproxy, and with the companion gem, using it in Rails apps is a piece of cake.

When it comes to styling emails, the premailer-rails gem is truly a gem of a gem. Use your CSS classes freely and let Premailer unwrap them into style attributes during email template rendering.

Crafting APIs (and GraphQL)

Even though the HTML-over-the-wire approach is gaining back its popularity, years of JS domination won’t be so easily undone. And so, API-first Rails applications don’t seem like they’re going anywhere.

When crafting JSON APIs, we usually use the following tools:

gem 'alba'
gem 'oj'
# or
gem 'panko_serializer'

Both Alba and Panko focus on data serialization performance and provide familiar (Active Model Serializer-like) interfaces. Panko is the fastest serialization library (due to its use of a C extension) amongst the currently maintained and closer resembles active_model_serializers API (so it’s a good candidate for migration). On the other hand, Alba is more feature-rich and pretty fast, too (when using Oj as a JSON serialization driver).

However, in practice, we more frequently build GraphQL APIs, and here are the relevant gems:

gem 'graphql', '~> 2.0.0'

# Cursor-based pagination for connections
gem 'graphql-connections'
# Caching for GraphQL done right
gem 'graphql-fragment_cache'
# Support for Apollo persisted queries
gem 'graphql-persisted_queries'

# Yes, Action Policy comes with the official GraphQL integration!
gem 'action_policy-graphql', '~> 0.3'

gem 'graphql-schema_comparator', group: :development

# Solving N+1 queries problem like a (lazy) pro!
gem 'ar_lazy_preload'

Most of the GraphQL libraries we use are performance-oriented (and Martian built …coincidence?) The graphql-schema_comparator gem is used along with Danger to warn pull request reviewers about schema modifications (so they can pay proper attention to the change.)

You may ask why we added the ar_lazy_preload gem to the GraphQL group. Despite being an Active Record extension, this library is especially handy in conjunction with GraphQL APIs, where classic N+1 busters like #preload and #eager_load are not so efficient. With lazy preload, we can avoid both performance regressions and any complexity overhead from using data loaders or batch loaders (although not completely, of course).

Logging and instrumentation

The production group of our mythical Gemfile contains logging and monitoring tools:

group :production do
  gem 'yabeda-sidekiq', require: false
  gem 'yabeda-puma-plugin', require: false

  gem 'lograge'
end

Yabeda is an instrumentation framework for Ruby and Rails apps. It comes with plugins for popular libraries (like Sidekiq and Puma) and monitoring backends (Prometheus, DataDog, etc.).

Lograge turns verbose Rails output into concise structured logs, which can be parsed by log collection systems and turned into queryable data sources.

Development tools

Let’s move from production to development—our actual work as engineers. Ruby is designed for developer happiness, but to make developing applications with Ruby follow through with this principle, we need to tweak our dev tools a bit. What makes a Rails developer really happy? Having zero trouble with the development environment 🙂 Docker solves this problem well enough. What else? Doing less boring work. So, we’ll add robots to help us with that so we can focus on cool things. Or, we add gems to make our developer experience more pleasant and help us write better code.

Below we list popular development tools used by Martians in Rails projects.

gem 'bootsnap', require: false

Bootsnap is a part of the Rails default bundle, so no introduction is required. Just use it!

gem 'database_validations'
gem 'database_consistency'

This pair of gems, database_validations and database_consistency, helps you to enforce the consistency between your model layer and the database schema. If it’s null: false, then it must have validates :foo, presence: true. If it validates :bar, uniqueness: true then a unique index should be defined for the column(-s).

gem 'isolator'
gem 'after_commit_everywhere'

Since we usually run on a transactional database, we should always remember about ACID. Database transactions are not the same as logical transactions, non-database side-effects are not protected by the database, and we should take care of them ourselves. For example, sending a “Post published” email from within a database transaction could result in a false positive notification in case the transaction would fail to commit. Similarly, performing an API request could lead to inconsistency in the third-party system (e.g., a payment gateway) in case of a rollbacked transaction. Isolator helps to identify such problems and after_commit_everywhere provides a convenient API to perform side-effects on successful COMMIT.

More database-related goodies with short annotations:

# Make sure your migrations do not cause downtimes
gem 'strong_migrations'
# Get useful insights on your database health
gem 'rails-pg-extras'

# Create partial anonymized dump of your production database to perform
# profiling and benchmarking locally
gem 'evil-seed'

Benchmarking and profiling is a part of the development process; here are some of the gems to help with it:

# Speaking of profiling, here are some must-have tools
gem 'derailed_benchmarks'
gem 'rack-mini-profiler'
gem 'stackprof'

Speaking of Stackprof, we can’t help but mention an awesome stack profiling reports viewer—Speedscope.

gem 'bundler-audit', require: false
gem 'brakeman', require: false

Security is a must. Check for any known CVEs in your dependencies by running bundle audit and scan your codebase for security breaches regularly with Brakeman. Nowadays, CI services provide their own security analysis tools which can be used instead.

gem 'danger', require: false

Danger can level up your code review experience by automating routine tasks and highlighting important changes: automatically attach labels, warn about missing tests or undesired structure.sql changes—delegate pretty much whatever you want to automation scripts.

gem 'next_rails'

Upgrading Rails? The next_rails gem can guide you through the process.

gem 'attractor'
gem 'coverband'

Keeping codebases in a healthy state requires us to perform regular check-ups and monitor the parts of your code that are hurting the most. You can use static analysis, for example, the famous churn/complexity technique to identify the components worth refactoring. There are plenty of tools for that, but our choice is Attractor. Production coverage (via Coverband) provides a different perspective on which code is more important (and thus requires more attention during refactoring); it also identifies any dead code to be eliminated.

Here’s the final chord in this little symphony:

eval_gemfile 'gemfiles/rubocop.gemfile'

# gemfiles/rubocop.gemfile
gem 'standard'
gem 'rubocop-rspec'
gem 'rubocop-rails'

You can learn more about our RuboCop-ing approach in the corresponding post: RuboCoping with legacy: Bring your Ruby code up to Standard.

Testing tools

Writing and running tests is also a part of everyday development, but it should still be treated with a personal touch. A highly advanced testing culture is one of the most significant benefits of building applications with Ruby and Rails.

Let’s start with the basics:

gem 'rspec-rails'
gem 'factory_bot'

Yes, we are RSpec fans, and we prefer factories (via FactoryBot ) over fixtures (but not exclusively). Let’s leave a heated discussion on Minitest vs. RSpec and Factories vs. fixtures for another day and continue with paradigm-agnostic libraries.

gem 'cuprite'
gem 'site_prism'

Crafting fast and robust system tests (or browser tests) is a combination of modern tooling to control the browser (Cuprite) and an object-oriented approach to describing scenarios (via site_prism’s page objects).

gem 'with_model'

The with_model gem is one of our favorites. I first discovered it many years ago when I was looking for a better way to test Rails model concerns. The library allows you to create temporary models backed by temporary database tables to test modules, Rails concerns, and any code implementing a shared behavior.

gem 'n_plus_one_control'

The N+1 queries problem is one of the most frequent issues with Rails applications: it’s so easy to introduce it, and there are some tools to detect it, but not to prevent it (except the recent .strict_loading Active Record feature). With n_plus_one_control gem, you can write tests to protect yourself from introducing N+1 queries in the future.

gem 'webmock'
gem 'vcr'

Tests should never access the outer world. My rule of thumb is as simple as this: a test suite is only good if I can run it while being on a plane and it passes. Simply dropping WebMock.disable_net_connect! to a rails_helper.rb or test_helper.rb can help you to avoid dependency on real network calls.

Network calls (and time, too) could also cause your tests to have flakiness:

gem 'zonebie'

One elegant way to prevent time-dependent tests is to run them in a random time zone. The zonebie gem does this by just being included in the bundle.

Note that we haven’t included the famous Timecop gem to control time: that’s because it’s redundant for Rails apps. Instead, you can use built-in time helpers.

gem 'rspec-instafail', require: false
gem 'fuubar', require: false

A couple of goodies for RSpec users: Fuubar is a progress bar formatter (why isn’t it the default?), and the rspec-instafail formatter is useful when running tests on CI (so you can see any errors right away and start fixing them before the entire test suite has finished).

And last, but not least, our final ingredient to our Gemfile’s test group:

gem 'test-prof'

TestProf is a “good doctor for slow Rails test suites”. It doesn’t matter if you have hundreds or dozens of thousands of tests, it’s always worth it to make them run faster. And with TestProf, you can drastically improve test run time with little effort.

TestProf: a good doctor for slow Ruby tests

TestProf: a good doctor for slow Ruby tests

August 15, 2017

Read also

Everything else

It’s basically impossible to describe all the awesome libraries we use in just a single blog post; so we picked a handful of bonus libraries to share here:

gem 'anycable-rails'

gem 'feature_toggles'

gem 'redlock'

gem 'anyway_config'

gem 'retriable'

gem 'dry-initializer'
gem 'dry-monads'
gem 'dry-effects'

Each one of the above answers a particular question:

• Want to build some real-time features? AnyCable is the key.
• Need to quickly introduce feature toggling? There is a minimalistic solution for that: feature_toggles.
• Looking for a distributed locking mechanism? Redis works great for that, and redlock is the way to go.
• Want to keep application configuration under control? Consider using configuration classes via Anyway Config.
• Tired of writing retry logic by hand? Take a look at retriable.
• Service objects a mess? Consider standardizing them with a sprinkle of dry-rb. For instance, declarative parameters from dry-initializer and/or Result objects from dry-monads (they work especially well with pattern matching).
• Want to move the context around abstraction layers in a safer and predictable way? Try algebraic effects via the dry-effects library.

I could go on and on with questions and answers like this, but I don’t want to be too pushy. Every engineer has their own ideal tool belt; try and build your own! Use other kits as a reference—not a source of truth.

Only in dreams

On that note, this is a good place to snap back to reality. Hopefully, you have some conception of what the “ideal” Martian Gemfile might look like, and with any luck, you’ve got some inspiration to dream a little on your own, too.

Evil Martians can help with your project and product design needs. So whether it’s the Rails project of your dreams (or something beyond) if you have a web or mobile application in need of expert problem solving with product design, frontend, backend, or software reliability, we’re ready! Or are you looking to get your project off the ground and into reality? Reach out to us!

Changelog

1.0.1 (2023-01-20)

• Replace redis-mutex with redlock (because even the author suggests this)