rowtheboat

Back in Feburary 2007 myself and Andy went to BarcampLondon2 on an exploratory trip to bash around a few ideas for what would later become idlasso. I presented on scaling web applications, the presentation for which can be seen at slideshare or below:

I’ve just found a great video of Google’s CIO on innovation. There’s a fantastic part where he states you need to reward innovation and risk taking and not punish failure. In my experience this seems to be the polar opposite from most companies these days!

I’m really into rails performance (and now merb but that wasn’t around when this post was written) so decided to look at a comparison between the standard rails versions - 1.1, 1.2 and 2.0. In these tests the excellent railsbench was used to drive the tests - I favor this over the standard rails benchmarking tools is it measures the raw performance of Rails request processing, ignoring the time spent passing the request from the web server to the Rails application. Additionally it is consistent across rails versions and I’m aware there have been changes in the benchmarking area in the rails 2.0 release.

The hardware

The tests were run on an old machine of mine dedicated to performance testing. It’s an AMD Athlon 1.4GHz, 256 MB RAM, Ubuntu 6.06 LTS server edition with latest security patches applied, Ruby 1.8.6 and Gems 1.0.1 (installed from source) and a dedicated crossover ethernet cable to a much more powerful desktop machine. It’s not the aim of these tests to find reasonable performance values for a modern server but more the most efficient rails version to use.

The tests

The tests compose of several pages designed to stress different areas of rails. These are:

In all tests (unless otherwise noted) 5000 requests were performed 3 times and the average request time taken.

In the first tests I performed it was noticed that creating actions in a web service style (respond to different MIME types) caused a significant performance hit. EG in rails 1.1 the code:

def index
  @users = User.find(:all)
end

was changed in rails 1.2 to:

def index
  @users = User.find(:all)

  respond_to do |format|
    format.html
  end
end

As a result two versions of the test appliaction were created, one with responds_to blocks and one without.

There were some changes in the edit view in each of the versions above. The first change was introduced in rails 1.2 due to a new way of using routes as a result of RESTful applications. So rather than:

<%= form_tag :action => 'update', :id => @user %>

in rails 1.2 we use:

<%= form_tag(user_url, :method => :put) %>

Testing showed this made no noticable difference to performance.

The second change was introduced in rails 2.0 as a result of a new way of constructing forms. So rather than:

<%= form_tag(user_url, :method => :put) %>
    &lt;%= text_field(:user, :notes) %&gt;
    &lt;%= submit_tag %&gt;
&lt;%= end_form_tag %&gt;

in rails 2.0 we use:

<% form_for(@user, :url => user_url, :method => :put) do |f| %>
    <%= f.text_field(:notes) %>
    <%= submit_tag %>
<% end %>

This is only really a style change from the rails 1.2 approach and testing showed it made little difference to performance.

The results

Showing requests per second:

Or in graph form (click for the large image):

So what can we see from the results?

• Rails 1.1 seems to excel when rendering a simple page with no session, but then you would sort of expect this as the codebase will be smallest. Still the margin was quite significant.
• Rails 1.1 and rails 1.2 share the honors for fastest at processing the page depending on the test (indeed their averages are very close).
• The web service versions of the rails 1.2 and rails 2.0 applications are on average 9.6% and 5.8% slower respectively
• If you’re jumping from Rails 1.1 to Rails 2.0 with web services (as we are going to at idlasso.com at some point) then you could see a performance decrease of 17.7% on average

Conclusion

So does this mean rails 2.0 is the slowest release yet? Well no. In order to keep the comparison fair the session store default in the rails 2.0 tests was changed to pstore which is the default used by rails in versions 1.1 and 1.2. However in rails 2.0 the default was changed to use a cookie stored in the user’s browser. A quick test using Apache Bench shows that rails 1.2 averaged 48req/s whereas rails 2.0 averaged 60req/s on the front2 page. However this was just that - a quick test. Only 1000 requests were sent, with no averaging or warmup. I’ll explore sessions more fully in a later article but for now we can safely say use cookie based session if using rails 2.0

So from this series of performance tests it seems it’s sensible to recommend:

• Avoid the use of the respond_to statement unless your action actually responds in more than one way
• Use client side sessions if using rails 2.0 or later.

PC-Rower was a piece of software produced for my dissertation at Durham. The idea was simple, plug rowing machines into a computer and create images of boats racing down a river that can be projected in front of the rowing machines. It was created using SWT from the eclipse project and communicated with the rowing machines through a serial port on the back of the PM2+. It worked really well and the Durham University boat club found it helped performance during training.

Unluckily as I was finishing the project Concept2 produced a new rowing machine with a USB performance monitor, a company founded and produced similar software (albeit only on windows), the Java support for USB sucked and I was off traveling for 3 months. Consequently the project died a premature death. I decided to open source the project and if nothing else has it provides a good implementation of a (reversed engineered) serial protocol.

Over time this will become a proper website, with access to all my digital stuff. Right now I’m quite tied up with work, two of my own projects and various other bits so a simple wordpress blog will suffice! I’m also going to back migrate posts from various efforts over the years so all my content is now in one place :)

Just a quick post to say we won £1000 at Barcamp Sheffield in the best new startup competition (see us get our cheque here). Thanks to all at PlusNet for a great weekend.