Ru: Ruby in Your Shell

ruby

The venerable sed and awk are extremely powerful text-processing tools. In the hands of a master, these power tools can bend text into almost any shape and form. Unfortunately, I’m no master. More importantly, I don’t really want to invest too much time into learning sed and awk (or even bash!), especially when most of my text processing tasks are ad-hoc.

Enter Ru. Ru let’s you, the lazy Ruby programmer, use your favorite programming language in the shell. This means that you can map, sum, and capitalize to your heart’s content. In this article, we will explore some of the nifty things that Ru lets you do, and how it can simplify your text-processing tasks.

Installing Ru

Ru comes packaged as a Ruby gem. To install Ru, fire up your terminal:

% gem install ru                                                            
Successfully installed ru-0.1.4
1 gem installed

The Basics

Before we get to the good bits, let’s learn a bit about how Ru lets us use Ruby to interact with the shell. After installing the gem, we can invoke Ru with the ru command. Similar to typical Unix commands, ru reads from standard input (STDIN), as well as files.

Here’s the first example file for today’s exploration:

% cat a_tale.txt
It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness, it was the epoch of belief, it was the epoch of incredulity, it was the season of Light, it was the season of Darkness, it was the spring of hope, it was the winter of despair…

ru can take a Ruby command and a file name as an argument:

% ru 'map(&:upcase)' a_tale.txt                                   
IT WAS THE BEST OF TIMES, IT WAS THE WORST OF TIMES, IT WAS THE AGE OF WISDOM, IT WAS THE AGE OF FOOLISHNESS, IT WAS THE EPOCH OF BELIEF, IT WAS THE EPOCH OF INCREDULITY, IT WAS THE SEASON OF LIGHT, IT WAS THE SEASON OF DARKNESS, IT WAS THE SPRING OF HOPE, IT WAS THE WINTER OF DESPAIR…

The file can also be passed along using pipes:

% cat a_tale.txt | ru 'map(&:upcase)'

In fact, it should be of no surprise that you can pipe the output of ru into another command. Suppose I wanted to count the words:

% cat a_tale.txt | ru 'map(&:upcase)' | wc -w
60

This is extremely powerful, since ru is just another Unix command that takes an input and manipulates its output.

With the basics out of the way, let’s head on to the fun stuff.

Text-Processing-Fu in Ru

The best way to get a feel of Ru is to see some examples, or better yet, try them out yourself! Here’s the example file that we will work with for the next few examples:

% cat chapters.txt
from zero to deploy
a toy app
mostly static pages
rails-flavored ruby
filling in the layout
modeling users
sign up
log in, log out
updating, showing, and deleting users
account activation and password resets
user microposts
following users

Filtering

Let’s say I want to search for chapter titles that might be too wordy:

% cat chapters.txt | ru 'select { |l| l.split.size == 5 }'
updating, showing, and deleting users
account activation and password resets

Internally, Ru reads the file as an array of lines. The select { ... } is then called on the array. Because it is an array, you can call any Array method.

Skipping Lines

Sometimes, it is useful know about line numbers. In this example, I want to skip odd lines, and remove any empty lines:

% ru 'map.with_index { |line, index| index % 2 == 0 ? line : "" }' chapters.txt | grep -v '^$'
from zero to deploy
mostly static pages
filling in the layout
sign up
updating, showing, and deleting users
user microposts

Once again, Ru doesn’t stop you from combining the output with existing commands. Here, we are using grep to filter out empty lines.

Titleize and ActiveSupport

Ru comes with ActiveSupport built-in. That is super sweet, because it adds handy functions, such as titleize. This is very useful for the occasional article writer who is absolutely clueless about the rules to titleizing headers:

% ru 'map(&:titleize)' chapters.txt
From Zero To Deploy
A Toy App
Mostly Static Pages
Rails Flavored Ruby
Filling In The Layout
Modeling Users
Sign Up
Log In, Log Out
Updating, Showing, And Deleting Users
Account Activation And Password Resets
User Microposts
Following Users

(OK, maybe it doesn’t do it exactly right, but it’s still cool…)

Unix VS Ruby

Say I wanted to find the top 10 commands I frequently use. In a completely Unix-land, I might do:

% history | awk '{print $2;}' | sort | uniq -c | sort -rn | head -10

This returns an output like:

1737 cd
1052 git
 880 ls
 857 vim
 608 gst
 310 mix
 263 exit
 248 gc
 240 fg
 224 iex

While my head can intuitively understand what is going on, I’m not too sure about all the additional flags needed. Also, I cheated and had to search StackOverflow for the above incantation.

Here is the equivalent in Ru:

history | ru 'inject(Hash.new(0)) { |h, l| h[l.split[1]] += 1; h }.sort_by { |k, v| -v }.map(&:first).take(10)'

Ruby is great because of its readable syntax and expressiveness, and the Rubyist in me understands perfectly what is going on, even if it seems slightly more verbose.

JSON Formatter

I’ve always had to prettify JSON at some point during web development. For example, take the following JSON:

% cat sample.json
{ "id": "0001", "type": "donut", "name": "Cake", "ppu": 0.55, "batters": { "batter": [ { "id": "1001", "type": "Regular" }, { "id": "1002", "type": "Chocolate" }, { "id": "1003", "type": "Blueberry" }, { "id": "1004", "type": "Devil's Food" } ] }, "topping": [ { "id": "5001", "type": "None" }, { "id": "5002", "type": "Glazed" }, { "id": "5005", "type": "Sugar" }, { "id": "5007", "type": "Powdered Sugar" }, { "id": "5006", "type": "Chocolate with Sprinkles" }, { "id": "5003", "type": "Chocolate" }, { "id": "5004", "type": "Maple" } ] }

Python comes with a nifty tool that does this:

cat sample.json | python -m json.tool

I’ve been suffering from some Python envy because of this. Let’s try to achieve a similar effect in Ru:

% cat sample.json | ru 'require "json"; JSON.pretty_generate(JSON.parse(files.join))'

This gives you the expected prettified JSON:

% cat sample.json | ru 'require "json"; JSON.pretty_generate(JSON.parse(files.join))'
{
  "id": "0001",
  "type": "donut",
  "name": "Cake",
  "ppu": 0.55,
  "batters": {
    "batter": [
      {
        "id": "1001",
        "type": "Regular"
      },
      {
        "id": "1002",
        "type": "Chocolate"
      },
      {
        "id": "1003",
        "type": "Blueberry"
      },
      {
        "id": "1004",
        "type": "Devil's Food"
      }
    ]
  },
  "topping": [
    {
      "id": "5001",
      "type": "None"
    },
    {
      "id": "5002",
      "type": "Glazed"
    },
    {
      "id": "5005",
      "type": "Sugar"
    },
    {
      "id": "5007",
      "type": "Powdered Sugar"
    },
    {
      "id": "5006",
      "type": "Chocolate with Sprinkles"
    },
    {
      "id": "5003",
      "type": "Chocolate"
    },
    {
      "id": "5004",
      "type": "Maple"
    }
  ]
}

There’s a couple of things to notice here. First, we can do require just like any Ruby script. Take a look how we managed to do the JSON prettification again:

% cat sample.json | ru 'require "json"; JSON.pretty_generate(JSON.parse(files.join))'

So far, we have assumed that the first argument is implicitly the file. However, in this case, the lines in the file are captured in the files method. We need to call join in this case because JSON.parse expects a String as input.

Obviously, typing so much just to get JSON prettification is a major hassle. Before you develop any Python envy yourself, Ru has a nice trick up its sleeve – ru save. With ru save , you can name a command and save it. For example, to save the JSON prettifier:

%  ru save jsonify 'require "json"; JSON.pretty_generate(JSON.parse(files.join))'
Saved command: jsonify is 'require "json"; JSON.pretty_generate(JSON.parse(files.join))'

Then we can run the command with ru run command:

%  cat sample.json | ru run jsonify

To get a list of saved commands, type ru list:

% ru list
Saved commands:
jsonify require "json"; JSON.pretty_generate(JSON.parse(files.join))

Ru Helps You Simplify Your Workflow

While I don’t use Ru that often, I’m glad that I always have it in my toolbox. To learn more about Ru, head over to the official page. There are some excellent examples showcased, such as:

Thanks for reading!