Fork me on GitHub
Minuteman

What is Minuteman?

Minuteman provides fast analytics using Redis as a backend. Uses bitwise operations to provide complex data analysis and reporting.

Wikipedia: Minutemen were members of teams from Massachusetts that were well-prepared militia companies of select men from the American colonial partisan militia during the American Revolutionary War. They provided a highly mobile, rapidly deployed force that allowed the colonies to respond immediately to war threats, hence the name.

What makes it so fast?

Redis makes it fast, stupid fast. The core concept of using bit arrays to store information plus the nuclear tornado that powers Redis makes the mixture a fast and furious way to store analytics.

I use Rails, is there a fast way to use it?

YES

What's the idea behind it?

It's all about the bytes, well, actually it's all about Redis. Since Redis 2.6 bit operations are part of this great db and that inspired different people. Thanks to them I've built Minuteman:

Getting started

How about complex operations?

This is the Venn representation of the operation, the power off Minuteman It's the ablitity to merge and substract groups.

What can I do if I want to join it with my User table?

Minuteman explained

Let's go through bit operations once again:

This is the number 15 represented in 8 bits, the key concept of this. But also we can see that as bits 0 to 3 are marked. Ignoring 0 we also can see this as user_id 1 to 3 are marked

Having our users represented as marked bits gives us access to bit operations:

AND: &

Now let's imagine our first bit array represents users who get an email invitation and the other set represents the users who bought our product. Then we can find out the intersection of them.

OR: |

We can imagine the same but joining two sets to do another operation afterwards

XOR: ^

The exclusive or aka XOR will behave like OR but ignoring the common bits.

NOT: ~

This will reverse the marked bits

Pictures from http://www.micahcarrick.com/tutorials/avr-microcontroller-tutorial/avr-c-programming.html