Benchmarking your Ruby with time_up

If you were losing a lot of time to distractions and wanted to start spending more of your day focused on what matters, you might start with a personal time-tracking tool like RescueTime, Toggle, or Daily, to measure and understand where your time was going.

When I’m working on an application, I often find myself wanting the same thing for my code: more time spent on what I care about, less time wasted repeating low-value activities. Sometimes, while waiting for a command to finish or for a test to pass, I’ll ask myself this seems too slow, where is all my computer’s time going?

We already have a lot of tools for similar tasks. If you already know which critical code path needs to be optimized, you could run Benchmark against it repeatedly in isolation. Or, if you want to scan a whole system for problem areas, you could use a profiler to shine a light on the most resource-intensive operations. And if you simply want to know which requests and queries are taking too much time, you could sign up for any of a dozen application performance monitoring services.

But what if you want answers to questions like these:

• Relatively speaking, does this API endpoint spending more time querying for data, transforming it, or responding over the network?
• How much time does the test suite spend creating and destroying factories in aggregate?
• How does a new XML parser perform against different sizes and types of files if you segregate unrelated operations like slow network requests or database inserts?

Questions like these can often be tricky to answer without custom timing code or invasive runtime introspection, either of which could unintentionally impact the behavior and performance of the system—the very thing you’re trying to measure!

So that’s why we made time_up, a new Ruby gem that helps you track where your code’s time is going. And it’s easy to use! If you’ve ever used a stopwatch ⏱, you’ll know exactly what to do with a mere skim of the README.

How to use time_up

After you’ve installed and required the gem, you’re ready to start measuring where your app’s time is going! One easy way to get started is to pass a timer name and a block to TimeUp.start, like so:

transactions = Transaction.where(user: @user)

# `TimeUp.start` will return whatever the block evaluates to
monthly_spending = TimeUp.start(:processing) {
  categorized_transactions = group_by_category(transactions)
  arrange_by_month(categorized_transactions)
}

payments = PaymentFinder.find(@user, monthly_spending)

render json: present_summary(
  spending: monthly_spending,
  payments: payments
)

When you call TimeUp.elapsed(:processing) you’ll get a Float back that represents how many seconds have elapsed while the :processing timer was running. Just like a real stopwatch, it will count up the total number of seconds elapsed while running, even if you start and stop it multiple times (that is, unless you call TimeUp.reset).

You can accomplish the same thing imperatively, and with a reference to the timer itself:

process_timer = TimeUp.start(:processing)
categorized_transactions = group_by_category(transactions)
monthly_spending = arrange_by_month(categorized_transactions)
process_timer.stop # Returns the total elapsed time in seconds

Most of TimeUp’s module methods are available on each timer instance, so process_timer.elapsed will also give you the result. The gem provides a number of methods for controlling timers (stop, start, reset), inspecting their status (active?, TimeUp.active_timers), and collecting basic statistics (like count, min, max, mean, median, and percentile). The full API is outlined in the README.

Managing multiple timers

But where time_up really excels is in timing different activities separately, so you can get a rough comparison of where your app is spending its time in relative terms.

It will look a little messy, but continuing this example, we could carve up our measurement by each category of the code’s behavior by using multiple timers:

transactions = TimeUp.start(:querying) {
  Transaction.where(user: @user)
}

process_timer = TimeUp.start(:processing)
categorized_transactions = group_by_category(transactions)
monthly_spending = arrange_by_month(categorized_transactions)
process_timer.stop

payments = TimeUp.start(:querying) {
  PaymentFinder.find(@user, monthly_spending)
}

render json: TimeUp.start(:presenting) {
  present_summary(
    spending: monthly_spending,
    payments: payments
  )
}

Whether you’re timing a script that’s only run once or invoking these blocks repeatedly in a long-lived process like a web application, all of the timers you create will be persisted in memory for the life of the process.

Analyzing your recordings

Once you’ve recorded some time, you’ll probably want to know how long everything took! As mentioned above, the gem provides a number of methods to inspect the timers that are available on both the TimeUp module and each timer instance:

> TimeUp.elapsed(:presenting)
=> 1.0050820000469685
> query_timer = TimeUp.timer(:querying)
> query_timer.count
=> 2
> query_timer.timings
=> [0.5051469999598339, 0.8050760000478476]
> query_timer.min
=> 0.5051469999598339
> query_timer.max
=> 0.8050760000478476
> query_timer.mean
=> 0.6551115000038408

The gem also provides a few roll-up methods across timers like TimeUp.total_elapsed, TimeUp.all_elapsed, TimeUp.all_stats:

> TimeUp.total_elapsed
=> 2.822688000043854
> TimeUp.all_stats
=> {
  :querying=>{:elapsed=>1.3102230000076815, :count=>2, :min=>0.5051469999598339, :max=>0.8050760000478476, :mean=>0.6551115000038408},
  :processing=>{:elapsed=>0.5073829999892041, :count=>1, :min=>0.5073829999892041, :max=>0.5073829999892041, :mean=>0.5073829999892041},
  :presenting=>{:elapsed=>1.0050820000469685, :count=>1, :min=>1.0050820000469685, :max=>1.0050820000469685, :mean=>1.0050820000469685}
}

Printing your results

Since it’s pretty common to want to print out the results, the gem provides a couple methods to quickly output the state of all your defined timers.

If you’re only executing the code that you’re timing once, you can print a simple summary of elapsed times with TimeUp.print_summary:

> TimeUp.print_summary

TimeUp summary
========================
:querying   	1.30826s
:processing 	0.51018s
:presenting 	1.00513s

And if you’re timing the behavior of code that’s called repeatedly (as in a controller action or a test helper), you can print a more detailed table that includes basic stats of all the timings recorded for each named timer with TimeUp.print_detailed_summary:

> TimeUp.print_detailed_summary

=================================================================================
   Name     | Elapsed | Count |   Min   |   Max   |  Mean   |  Median  |  95th %
---------------------------------------------------------------------------------
:querying   | 1.31028 | 2     | 0.50520 | 0.80507 | 0.65514 | 0.65514  | 0.78221
:processing | 0.50314 | 1     | 0.50314 | 0.50314 | 0.50314 | 0.50314  | 0.50314
:presenting | 1.00508 | 1     | 1.00508 | 1.00508 | 1.00508 | 1.00508  | 1.00508

If you’re planning to intersperse your usage of TimeUp throughout a long, messy process like running all your tests, it might be easiest to print your results in an at_exit hook. And if you want to output them to a custom logger, both methods accept an IO argument.

Have a great time!

The time_up gem is a simple tool for a simple job. Hopefully you’ll remember it the next time you find yourself wondering which parts of a complex operation are taking the most time.

If you have any questions or ideas, feel free to open an issue or hit us up on Twitter!