rowtheboat - George Palmer

I’m really into rails performance (and now merb but that wasn’t around when this post was written) so decided to look at a comparison between the standard rails versions - 1.1, 1.2 and 2.0. In these tests the excellent railsbench was used to drive the tests - I favor this over the standard rails benchmarking tools is it measures the raw performance of Rails request processing, ignoring the time spent passing the request from the web server to the Rails application. Additionally it is consistent across rails versions and I’m aware there have been changes in the benchmarking area in the rails 2.0 release.

The hardware

The tests were run on an old machine of mine dedicated to performance testing. It’s an AMD Athlon 1.4GHz, 256 MB RAM, Ubuntu 6.06 LTS server edition with latest security patches applied, Ruby 1.8.6 and Gems 1.0.1 (installed from source) and a dedicated crossover ethernet cable to a much more powerful desktop machine. It’s not the aim of these tests to find reasonable performance values for a modern server but more the most efficient rails version to use.

The tests

The tests compose of several pages designed to stress different areas of rails. These are:

Page	Action	Area
front1	Renders current time without a session	Designed to hit as little of rails as possible
front2	Renders current time with a session	Designed to show the difference in using a session
users	Lists of users (100 in test database)	Designed to test the database driver
showuser	Shows the user details (also shows the user’s posts)	Designed to behave like a typical test page with a user who has_many posts
edituser	Thed edit user page (posts aren’t included)	Tests page rendering and form helpers
updateuser	Updates the user details via a POST	Accessing/Updating 1 database row

In all tests (unless otherwise noted) 5000 requests were performed 3 times and the average request time taken.

In the first tests I performed it was noticed that creating actions in a web service style (respond to different MIME types) caused a significant performance hit. EG in rails 1.1 the code:

def index
  @users = User.find(:all)
end

was changed in rails 1.2 to:

def index
  @users = User.find(:all)

  respond_to do |format|
    format.html
  end
end

As a result two versions of the test appliaction were created, one with responds_to blocks and one without.

Test name	Rails setup
performance11	Rails 1.1
performance12	Rails 1.2 without responds_to
performance12	Rails 1.2 with responds_to
performance20	Rails 2.0 without responds_to
performance20	Rails 2.0 with responds_to

There were some changes in the edit view in each of the versions above. The first change was introduced in rails 1.2 due to a new way of using routes as a result of RESTful applications. So rather than:

<%= form_tag :action => 'update', :id => @user %>

in rails 1.2 we use:

<%= form_tag(user_url, :method => :put) %>

Testing showed this made no noticable difference to performance.

The second change was introduced in rails 2.0 as a result of a new way of constructing forms. So rather than:

<%= form_tag(user_url, :method => :put) %>
    &lt;%= text_field(:user, :notes) %&gt;
    &lt;%= submit_tag %&gt;
&lt;%= end_form_tag %&gt;

in rails 2.0 we use:

<% form_for(@user, :url => user_url, :method => :put) do |f| %>
    <%= f.text_field(:notes) %>
    <%= submit_tag %>
<% end %>

This is only really a style change from the rails 1.2 approach and testing showed it made little difference to performance.

The results

Showing requests per second:

	front1	front2	users	showuser	edituser	updateuser	all
rails11	390.49	179.45	111.62	122.61	180.11	107.24	149.53
rails12	306.51	192.66	127.29	129.27	160.43	106.72	151.33
rails12ws	258.02	172.83	117.16	118.31	144.60	101.01	138.04
rails20	292.12	186.00	93.63	119.84	142.77	101.93	134.41
rails20ws	261.40	173.68	89.94	113.96	134.22	97.03	127.03

Or in graph form (click for the large image):

So what can we see from the results?

• Rails 1.1 seems to excel when rendering a simple page with no session, but then you would sort of expect this as the codebase will be smallest. Still the margin was quite significant.
• Rails 1.1 and rails 1.2 share the honors for fastest at processing the page depending on the test (indeed their averages are very close).
• The web service versions of the rails 1.2 and rails 2.0 applications are on average 9.6% and 5.8% slower respectively
• If you’re jumping from Rails 1.1 to Rails 2.0 with web services (as we are going to at idlasso.com at some point) then you could see a performance decrease of 17.7% on average

Conclusion

So does this mean rails 2.0 is the slowest release yet? Well no. In order to keep the comparison fair the session store default in the rails 2.0 tests was changed to pstore which is the default used by rails in versions 1.1 and 1.2. However in rails 2.0 the default was changed to use a cookie stored in the user’s browser. A quick test using Apache Bench shows that rails 1.2 averaged 48req/s whereas rails 2.0 averaged 60req/s on the front2 page. However this was just that - a quick test. Only 1000 requests were sent, with no averaging or warmup. I’ll explore sessions more fully in a later article but for now we can safely say use cookie based session if using rails 2.0

So from this series of performance tests it seems it’s sensible to recommend:

• Avoid the use of the respond_to statement unless your action actually responds in more than one way
• Use client side sessions if using rails 2.0 or later.

1 comment

PC-Rower was a piece of software produced for my dissertation at Durham. The idea was simple, plug rowing machines into a computer and create images of boats racing down a river that can be projected in front of the rowing machines. It was created using SWT from the eclipse project and communicated with the rowing machines through a serial port on the back of the PM2+. It worked really well and the Durham University boat club found it helped performance during training.

Unluckily as I was finishing the project Concept2 produced a new rowing machine with a USB performance monitor, a company founded and produced similar software (albeit only on windows), the Java support for USB sucked and I was off traveling for 3 months. Consequently the project died a premature death. I decided to open source the project and if nothing else has it provides a good implementation of a (reversed engineered) serial protocol.

No comments

Over time this will become a proper website, with access to all my digital stuff. Right now I’m quite tied up with work, two of my own projects and various other bits so a simple wordpress blog will suffice! I’m also going to back migrate posts from various efforts over the years so all my content is now in one place :)

No comments

Just a quick post to say we won £1000 at Barcamp Sheffield in the best new startup competition (see us get our cheque here). Thanks to all at PlusNet for a great weekend.

No comments