When I built the billing process for Travis CI's commercial offering, I decided to try out some new things to avoid callbacks in ActiveRecord models, including validations.

In 2010 I wrote about why callbacks and validations scattered about the persistence layer bother me. I recommend reading it to get the full background on this.

What I went for this time was a mix of a service layer that handles all the business logic and a layer of form objects that handle communications between the controller and the services, including handling validations.

The goal was to have simple Ruby objects to take care of these things. No special frameworks required. Inspiration in part stemmed from Django's form objects, though my implementation lacks the part that talks directly to the model, for instance to save data to the database. Quite intentionally so, as that part is up to the services layer.

The last thing I wanted to avoid is having to use attr_accessible in the persistence layer. In my view, that part is not something persistence should be concerned with. It's a contract between the controller and the services it calls into to make sure parameters are properly narrowed down to the set required for any operation.

Form Objects

For form objects, I looked at Scrivener, which was a great start. It's a very simple framework for form objects, the code could barely be simpler, but it lacks some validations, as it implements its own set.

On top of that, it doesn't tie in with Rails' form handling that well, which requires some parts of ActiveModel to work properly. Scrivener is great when you integrate it with e.g. Sinatra and your own simple set of forms.

It's so simple that I decided to take its simple parts and merge it with ActiveModel's validations support. Thanks to Rails 3, that part has been extracted out of the ActiveRecord code and can be used for anything.

The beauty of form objects is that they allow you to specify different views on the same data. Every database record wrapped by ActiveRecord can have multiple representations depending on which data is required by a specific form.


Here's the base code for the forms, which doesn't have a name, it's just a snippet of code that's part of our Rails project:

It defines a few things that are required by Rails' form_for, but other than that it's straight-forward. It can populate form attributes based on the model handed in, which makes it suitable for re-use, for instance when editing an existing object or when validations failed on update.

Here's a sample form object:

It declares a few attributes and some validations. Thanks to ActiveModel you could use anything provided by its validations package in a form object.

By declaring the attributes a form object brings a simple means of implementing mass assignment protection without requiring any sort of sanitization and without poisoning the model with attr_accessible and jumping through hoops in tests to create valid objects to work with.

If an attribute assigned to the form doesn't exist, the assignment will fail.

In the controller...

The interaction with the controller is rather simple, no added complexity:

I'm liking this approach a lot, and it's been in use for a few months. There'll be some refinements, but the simplicity of it all is what I find to be the best part of it.

It's all just plain Ruby objects with some additional behaviours. Add a simple service layer to this, and cluttered code in the ActiveRecord model is nicely split up into lots of smaller chunks that deal with very specific concerns.

Tags: rails, web

Wherein I write about Rails' current implementation of logging and ActiveSupport's greatest feature that was added in 3.0.

I've been thinking a lot about logging lately. I'm a big fan of logging. I also spent some quality time with a Rails app again recently. I'm not a big fan of Rails' logging. Something needs to change, either me or Rails' logging. I opted for the latter.

After some digging in I found that Rails 3 has improved quite significantly on the logging front. The output is still the same unparsable mess it used to be, but the way logging is implemented has changed quite drastically, and much for the better, I'd argue.

With Rails 3, one important thing was added, that drives parts of the both logging and benchmarking (e.g. for ActiveRecord's query measurements). Everything is now built around notifications and instrumentation. When ActiveRecord fires a query, it measure the time required and then triggers an event with the recorded time. Whether someone picks up the event is not of its concern. Which is exactly what the new notifications are about: separation of concern.

I was surprised to find that not a lot of people seemed to know about them. Let's have a short look at what it allows you to do.


The idea is far from new. The Pragmatic Programmer talked about blackboards, a similar mechanism, Cocoa has had something similar for ages, and now Rails has something like it too.

The basic idea is that you have a centralized repository where you can subscribe to events, for example to the event ActionController triggers when it process an action. Here's the code to extract the path that was requested by a user.

ActionSupport::Notifications.subscribe('process_action.action_controller') do |event|
  puts event.payload[:path]

Any number of subscribers can be attached to a message, they'll get notified. The default implementation is synchronous, but nothing could and should keep you from adding an implementation that uses a message queue instead.

This is pretty cool. I started using it to track metrics and not clutter the code with the specifics. When something of interest happens, an event is triggered. If someone listens, cool, if they don't we'll keep going. I could have one subscriber that collects metrics and another one for tracer logging.

Rails uses these notifications all over the place, in particular for logging. To avoid having lots of manual subscriptions to specific events, Rails also added a mechanism to subscribe to events for the sole purpose of logging them and without adding subscriptions manually for all of them. The LogSubscriber was born.


LogSubscribers are exactly that: easy ways to subscribe to events whose purpose is logging. Of course what you do with the events is up to you, but that's their main purpose. Every Rails component uses them, and every component has its own implementation of a LogSubscriber. Here's an excerpt of the one used by ActionController.

Every public method defined, except for logger, will be attached to an event of the same name. So the three relevant events for this subscriber are start_processing and process_action.

To make the improvement of notifications and log subscribers more visible, here's the code that did the same in Rails 2.3. You decide which you like better. I certainly enjoy the decoupled-ness of the log subscriber a lot more.

When you defined your LogSubscriber, you can attach it to a namespace:

ActionSupport::LogSubscriber.attach_to :action_view

This creates a new instance of the LogSubscriber and attaches all methods to their corresponding events. The event names are along the same lines as the example with process_action.action_controller. First the event's name, then the component's name.

Rails' Logging

The example above nicely brings me to Rails' logging. First up: I like Rails' idea of logging everything, for development purposes it's awesome and pretty helpful. When things are moved into production the fun stops for me though. The logging out put is hard to parse, and it's hard to make sense of because it's usually multiple lines per request.

Rails 3.2 recently added tagged logging and a request identifier to work around that. But that still doesn't solve the problem of the output in general being too noisy and hard to parse for a centralized logging service, or any logging service. If you don't care about your logs then I'm sure you're fine, but I care a great deal about my logs. When things break, they're the sole source of truth, and I like to make sure they're valuable enough to fulfill that premise. Rails' request logging gets in the way of that for the reasons outlined above.

To remind you of what we're talking about, here's an example log output for a single request:

Started GET "/" for at 2012-03-10 14:28:14 +0100
Processing by HomeController#index as HTML
  Rendered text template within layouts/application (0.0ms)
  Rendered layouts/_assets.html.erb (2.0ms)
  Rendered layouts/_top.html.erb (2.6ms)
  Rendered layouts/_about.html.erb (0.3ms)
  Rendered layouts/_google_analytics.html.erb (0.4ms)
Completed 200 OK in 79ms (Views: 78.8ms | ActiveRecord: 0.0ms)

It reads pretty nicely for sure, but as soon as you add more processes that dump their output in the same log, things get mingled and some of the information is, in my opinion just not necessary in production.

So we have a pretty centralized approach to logging, and me wanting to do something to improve the logging. Clearly the two could be made to work together.

Towards a Better Logging (in Production)

My ideal request logging is a single line per request, nothing more. That's clearly at odds with the output above, but thanks to the fact that (almost) everything is wrapped in log subscribers. Here's a line of something that would fit my purpose pretty nicely:

GET / format=html action=home#index status=200 duration=58.33 view=40.43 db=15.26

It's one line, it contains all the relevant information, it's pretty easy to parse for a machine, and it's easy to read for the human eye. Some told me the latter shouldn't be necessary, and I'm certainly not hung up on it, but I like to be able to skim logs.

The message is pretty clear: the HTTP method, the URL and an slew of optional parameters. If there's an exception that too should end up as single line. You can log the stacktrace of course, but it's much harder to make sense of for the same reason multiple lines of log output are hard to make sense of. They drown in a river of log output from multiple sources. I thought about only storing relations to an exception in the logs instead, e.g. an identifier or an error code. Throw in a request identifier as metadata stored with the exception, and you've got yourself a nice way to correlate exceptions and log lines.

How can we get to the output above? It turns out, it's actually pretty simple. We need to unhook the log subscribers for ActionView and ActionController events and hook in our own. The result is Lograge, a logging experiment I extracted into a library from the Travis CI source code, where I first started playing with the ideas I had around logging.

It adds its own log subscriber, discarding all irrelevant events, only accepting two events instead of the whole bunch included by default. The result is a single log line. Easy on the eyes, easy on the machine.

An Experiment in Logging

My main goal is to eventually have a saner way of logging requests in Rails (or any web framework, for that matter). Lograge is the beginning of that. I already got great feedback on the general idea and on specifics like the log output.

I have also yet to solve how to properly log request parameters, so this is only the beginning. I care a great deal about logging and I'd like to see that eventually improve in Rails so that other people start caring for their logs too, if only in production mode or as an optional feature. If not, Lograge will be here for you. I also have Rails 2.3 on my radar if you're still using that. It's a lot messier to implement, but not impossible.

If you want to know a bit more about the internals, the README is a good place to start. If you have any input on the ideas implemented in Lograge, the log output or anything else, feel free to open an issue. Let's talk about logging, and let's make it better.

Tags: rails, logging

Recently I've been having a foul taste in my mouth, or just a bad feeling, if you will. Whenever I started adding validations and callbacks to a model in a Rails application. It just felt wrong. It felt like I'm adding code that shouldn't be there, that makes everything a lot more complicated, and turns explicit into implicit code. Code that is only being run depending on the persistence state of an object. Code that is being hard to test, because you need to save an object to test parts of your business logic. Don't get me started on observers, I never was a fan of them. Putting stuff that should be run when an object was saved elsewhere is the worst kind of hiding business logic.

The two are somewhat bound together: Create an object, run a callback and validations. Update an object, check if an attribute is set, run a different callback. I started to loathe the lack of implicitness that callbacks gave my code, and I started to loathe that libraries everywhere copied that behaviour from ActiveRecord, and that you involve testing your database in your unit tests, though implicitly, and that noone had a better answer for how that stuff should work. Sure, it's not big news that involving your database in your unit tests is not a recommended practice, hence the overuse of mocking and stubbing fostered by RSpec, but it took a while for me to really feel the pain.

Ingredients: REST, Rails, Resources

The root problem (in a very philosophical way and in my very own opinion) lies in Rails' understanding of RESTful resources. The path to a resource is supposed to be thin, and a resource is mostly represented by something that's stored in a database. So far, so good. That thin layer however, the controller, implies that running any logic on a resource somehow ends up in simply updating the resource, setting some attribute signaling that an action needs to be taken. That's how tools like resource_controller, make_resourceful and inherited_resources came about, to get rid of the thin-layered controller tier, the boiler plate code. And I loved them, because I hated writing the same code over and over again.

But over time I realized that controllers aren't the problem, it's the way Rails treats resources that lead us to this mess where everything's declared in a model's class body, where you have to look at callbacks to find out what's going on when you create a new object, and where you clutter your models with code that just doesn't need to be there (sending mails, for example, or doing custom validations that are bound to specific state). It all started to lack explicitness for me. There's no clear path of entry, no list of method calls you can follow along.

I'm not sure how other people see this, obviously a lot of people don't realize it's a problem, or they're simply fine with how things work, I was for a long time too, almost five years now, to be exact. I know I'm not alone in this though, talking to others made me realize they're on a similar page, but given that it's the way Rails works, what can you do?


What about validations? I can live with validations being part of the model, but they're still part of the problem generated by callbacks, simply because validations are callbacks themselves. I always found myself ending up writing custom validations in before_validation et. al. hooks, and it just felt wrong, because the list of callbacks grew noticably. Validations are not always the same depending on your object's state. They can even be quite useless when you're handling a lot of backend code that's asynchronous and decoupled from the web interface.

What do we do to work around it? We resort to using save(false), whose lack of explicitness I cannot even begin to fully explain in a peaceful manner. Validations tend to depend on the object's state, at least more complex ones. They're not universal, they can be different when creating and saving objects. Yes, I know that in ActiveRecord you can say :on => :create, but for me that just emphasizes my point. Persistent state determines callbacks and therefore business logic, decreasing expressiveness.

Forms? Presenters?

I've looked into how Django is dealing with this, and it sort of struck a chord with me. Their approach is to define form objects that do the validations for you. They're somewhat decoupled from your business logic and are your intermediate layer between the web interface and the business model. Good lord, did I just recommend we all use presenter objects, even though Jay Fields said they don't work? I think I did, Bob!

It's not the way, but it's one way, and there's been work on bringing this to Rails, have a look at Bureaucrat for example. Again, it's one way to try solving the problem, it doesn't come without any downsides too. Tying the idea of explicit services to form data may or may not make unit testing harder, or at least mix two different concerns (forms and creating/manipulating objects). What I like about it though is that it decouples validation from the business logic, and ties it to the layer it's really necessary, the view.

Not without irony, I've started using something similar to handle embedded objects on a project using CouchDB. I wanted to extract out the stuff handling with embedded objects, because I was simply to lazy to add support for that to CouchPotato. The result felt amazingly good, almost like the above forms mixed with a *gasp\* service.


I'm still seeking for the answer to that one. Pat Maddox and James Golick suggest services as one way to make callbacks explicit, be sure to read through the comments on James' post. Now, I don't know about your history, but coming from a Java background that struck a bad cord with me. I laughed off the idea, to be frank.

One thing still bugs me about the approach James is laying out in his post, it's the extensive use of mocking. It sure does make for fast unit tests, and I'm starting to see the sense in it when using a service model, but I've grown fond of the fakes approach, which is why we wrote Rocking Chair, and why people came up with things like SuperModel, in-memory database implementations you can easily switch on and off, not exactly decoupling your unit tests from the persistence layer, but decoupling them from having a database running, making it easier to e.g. split batches of tests across several cores and processors, and simply making them faster.

However, the more I thought about it, the more sense the services approach made. You don't have to necessarily make it a separate class, depending on how much you like having all of your business logic in the model. I'm not sure which approach I'd prefer, because honestly, I haven't tried it yet, but I'm willing to give in. I'm willing to try things that let me ditch callbacks and validations, because I simply loathe them, they just don't feel right to me anymore when used in the way ActiveRecord does.

With Rails 3 for example, you can put your validations anywhere, they're not just bound to ActiveRecord anymore, they leave a lot more room for you to use them in different ways. I'm not suggesting you should get rid of validations, they're just a part of the callbacks madness, and they tend to not always be the same depending on what you're doing in a particular use case. Using save(false) just started to feel like a bad workaround to me to avoid having callbacks run. I want to save an object no matter what, and run whatever code I require in that course of action, in an explicit fashion.

Are Fine-grained Resources the Future?

The question that's left for me, and this is where I'm starting to gasp question Rails' approach to resources. I'm starting to think they're wrong, and that for the service or the forms model to work, the idea of a single resource you update whenever you want your model to do something needs to change. Having very specific services and their methods would mean a switch to more lightweight and finer-grained resources, specific resources where you e.g. accept an invitation not by PUTting to /invitation/12345, but instead to /invitation/12345/accept.

But, and here's the kicker, to me this leaves the question of how RESTful that approach is. This is where I'd like to get input. How do people using services already solve this problem? Have lots of tiny methods in your controllers? Maybe it's time to work more with Sinatra to get different ideas in my head, to rethink resources. I'm still not certain what a good and RESTful way to approach this would be. Whatever it is, it'd leave my business logic in a way that's a lot more approachable both for people reading it, and for my tests. That's something pretty dear to me, and more important than fully conforming to a REST model. There must be something better, because the current status quo of handling business logic with Rails' resources just doesn't work for me anymore, it doesn't make me a happy programmer.

The answers have yet to be found. My intention is not to rant, but rather to start a discussion around it. In my opinion the way Rails recommends handling resources doesn't have a future, if only for me, and even if that means using other frameworks in the future. I don't think we wasted time with Rails' current way, instead we're going to see an evolution, and I find that pretty exciting, because the future has yet to be written in this regard, and we can be a part of that.

Tags: rails, rest

I've been spending some quality time with two of my new favorite tools lately (CouchDB and Redis, duh!), and while integrating them into Scalarium some needs and as a result some smaller hacks emerged. I don't want to deprive the world of their joy, so here they are.

First one is a tiny gem that will allow you to use Redis as a session store. What's so special about it, there's redis-store, right? Sure, but I couldn't for the life of me get to work reliably. Seems that's due to some oddity in Rack or something, at least that's where my interest of further investigating the issues faded, and I decided to just rip the code off MemCacheStore, and there you have it, redis-session-store. Rails-only and proud of it.

While working on it I constantly kept a monitor process open on Redis. Great feature by the way, if not awesome. I used telnet, and somehow I constantly managed to hit Ctrl-C in the terminal I had the telnet session open in. Reconnecting manually is tedious, so I give you my little redis-monitor script:

Incredibly simple, but saves those precious moments you'd waste typing everything by hand.

Last but not least, here's a hack-ish patch to make CouchPotato (great CouchDB Ruby library by the way) dump view queries into the log file. The ugly part at the end is me trying to get the log that's output at the end of each request to include the DB time for CouchDB queries.

It's not great, but works for now. We'll very likely include something decent into CouchPotato without hacking into ActionController like that. Unfortunately to get this far, there's really no other way. I tried faking ActiveRecord, but you open a whole other can of worm doing that, because a lot of code in Rails seems to rely on the existance on the ActiveRecord constant, assuming you're using the full AR stack when the constant is defined. Here's hoping that stuff is out the door in Rails 3. Haven't checked to be honest.

Dump that file into an initializer, and you're good to go (for the moment).

Tags: rails, redis, couchdb

I've started a new blog (don't worry, I have not abandoned this one), specifically targeting experiences, wtf's and workarounds for the latter on Amazon's Web Services. I've been working quite a lot with them recently, and there's heaps of dark corners around, and things you need to be aware of. That's what that blog is about. Head on over to Daily AWS Wtf and give it a read. It's powered by the people of Peritor, so there'll be other people writing too.

The annual one and only German Rails Conference (some might call it Rails Konferenz) is upon us. I'll be speaking about legacy code, and I'll also be giving a tutorial on refactoring Rails applications, together with Jonathan Weiss. It's a great one day conference with a German audience. Great occasion to meet the German Rails scene. Hope to see you there!

Tags: rails, aws

I recently had to improve the breadcrumbs situation on one of my projects. Until now, breadcrumb definition were sprinkled across the controllers. While that was okay for a while, the current breadcrumbs have become a bit more complex, and I wanted to get them out of the controllers. One option would've been to put them into the model, but seriously, who does that? Plus, I have a lot of breadcrumbs that are not exactly related to a model, and if they are, it's not always an ActiveRecord model.

Instead I wanted a simple configuration file. I started out with a combination of arrays and hashes put into global variables, but that turned out to be a mess. What I really wanted, was something like this:

There, much nicer. In the view, all I really wanted to do was, instead of manually assembling all the crumbs in place:

<%= crumbs %>

What evolved naturally was a simple API and with a potential plugin, so there you go. It's called Crumble and lives on the GitHubs. You can read all the glory details on its usage in the README. I'm sure to add some more stuff, but for the moment I wanted to keep it simple, and have it support the needs of the project. If you have something to add, fork away.

Tags: rails, plugin

Next week I'll attend and speak at the RailsWayCon conference in Berlin. You can see me speak at the tutorial "Deploying Rails Applications", together with Jonathan Weiss, and talk about asynchronous processing in Rails applications. The line-up looks pretty good. See you there!

Apart from the awesome new features for users of the framework, Rails 2.3 got a lot of love on the inside too. It's no secret it's running on Rack now, and that switch made some of the internal code a lot easier on the eyes. They also added a Rails-internal middleware stack, on which some of the framework's functionality builds already.

When you go to the console and enter


you'll get of all the classes and blocks registered in the stack. Even ActiveRecord hooks into the stack to do minor connection cleanup after a request. Coming from the Java world it's pretty much comparable to the good old Servlet filter (I know they were gross, but still kinda cool), code that gets executed before and/or after each request. An application-wide around filter if you will.

If you look through the code of the classes listed here you'll see one common interface which stems from Rack. They all implement a method called call which gets only one parameter, and an initialize method which gets an application object.

The really cool thing about it is this: Where you had to resign to using the revolting alias_method_chain before, you can now just hook into the request chain wherever you want, without modifying the stack with awkward methods that make debugging a pain in the bum.

Now, where does that make run_later any cooler? Well, it doesn't, but in earlier version I needed to do an awkward thing that only affected development mode, where Rails unloads all classes after each request. run_later runs code in a separate thread, and depending on how long that code runs the classes would be unloaded when they're still accessed from the worker.

In earlier version I turned to, drumroll, alias_method_chain for that, now it's nothing but a simple class that hooks into the internal middleware stack, and delays the request until the worker finishes, or until a certain amount of time has passed (default is 10 seconds). I would argue that you just shouldn't run code taking that long using run_later, so it shouldn't be a major issue. Longer-running tasks like these should be run using a slightly more advanced and reliable mechanism.

Anyway, let's have a look at the code:

The middleware is implemented using a simple stack, unsurprisingly called MiddlewareStack. There you can hook into the chain of classes pretty much wherever you want. I simply append my class at the end, because I definitely want to run the code before ActionController's Reloader class gets to run and unloads the classes. But you can also specify a specific class where you want to hook into the chain, you can even swap existing handlers with your own. Pretty neat stuff. I'd highly recommend playing with it, in some places providing a filter on the middleware level makes more sense than putting the code into a controller-based filter.

Tags: run_later, rails, rack

On the heels of RailsConf Europe's cancellation comes the call for papers for one of the viable alternatives, the RailsWayCon. It's happening from May 25th to 27th in Berlin, so other than going to Scotland on Rails like yours truly, this is the conference to go to.

But most importantly, you should hand in a submission for a talk or a tutorial, you know you want to. You can find all the nitty gritty details on the conference's call for papers page. Deadline is February 20th, so hurry!

See you there!

I wanted to play the field of continuous integration tools a little bit more, so I finally gave Integrity a go. Its promise of being lightweight sure was tempting.

It's a small Sinatra app, and therefore should've been easy to set up using Passenger. The Integrity team recommends using nginx and Thin. Though I'm rather fond of nginx, I don't see any point using a different setup just for my CI system.

Getting it up and running is rather straight-forward. You create your local Integrity template directory using integrity install /path. For this to work with Passenger you also need a directory public in that directory, so if you create that you can just throw the usual Passenger virtual host block into your Apache configuration, the document root pointing to the freshly created public directory, and you're good to go. In the current master, there's already a fix for this issue, and running integrity install will create the public directory for you.

I have some gripes with Integrity though, one of them being that configuring notifiers for projects currently seems to be broken. It's sort of a big deal to me, because continuous integration lives from the team receiving notifications.

But otherwise, do give it a go, it's pretty slim and pretty slick too, though it doesn't yet have asynchronous building. It needs some sort a hook, e.g. a GitHub one to run automatically. There's also a demo available for your viewing pleasure.

Update: The issue with notification configuration not being saved seems to be resolved in the current master. It's not perfectly fixed, but at least now I can have Integrity notify me through Twitter. So if you need to, fetch the master, and build your own gem. Remember to rename it to 'foca-integrity' in the Rakefile's Jeweler configuration, otherwise it won't really catch on.

This post has been lying in my drafts folder for a while now, and since I'm trying out new approaches to shrink oversized controllers, it's about time to put this out, and get ready for describing alternatives.

One of the basic memes of Rails is "Skinny controller, fat model." And even though most examples, especially the ones using the newer REST features in Rails advertise it, there are still heaps of controllers out there that have grown too big, in both the sense of lines of code and the number of actions. I've had my fair share of super-sized controllers over the last months, and it's never a pleasant experience to work with them.

If you don’t believe me, look at the Redmine code. Whether it's laziness or lack of knowing better doesn't really matter. Fact is, if it keeps growing you'll have a hard time adding new features or fixing bugs. Error handling will become more and more cumbersome the more code you stuff into a single action.

And if it's pretty big already, chances are that people will throw in more and more code over time. Why not? There's so much code in there already, what difference does it make? Broken windows are in full effect here. If the controller is full of garbled and untested code already, people will add more code like that. Why should they bother writing tests for it or refactoring it anyway? Sounds stupid, but that's just the way it is.

On a current project I refactored several (and by that, I mean a lot) of these actions to merely four to ten lines of code. The project is not using RESTful Rails, but it wouldn't make much of a difference anyway. I've made some experiences that worked out pretty well for me, which would very likely help to make a controller RESTful. But that wasn't really the main objective on my current project. If they're still up to par when fully using REST I'll leave up to you to decide or, even better, update.

I’m not going to put code into this article, since most of the stuff is graspable without looking at code. If you see the sort of code I’m talking about you’ll understand.

It's actually just a few simple steps, but they can be both frustrating and exhausting, even when you take on step at a time (which you should, really).

Understand what it does

It's too obvious, isn't it? But still, a beast consisting of 160 of code should be approached carefully. Understand what each line does, and more importantly, why it does it. In an ideal world you can just read the code, and understand what it does, but we both know that’s wishful thinking. If you’re lucky you can just ask someone who knows their way around. But oftentimes you’re out of luck, and just have to gather as much information from the code as you can, or from playing around with the application itself.

Don't just look at the code, run it from the application, look at the parameters coming in from the views. Annotate the code if it helps you, it might help others as well.

Look at the view code too. This also isn't always a pleasant experience, but you will find hidden fields, parameters set and handed through in the weirdest ways for no apparent reason.

Test the hell out of it

Most likely the action at hand does not have any tests available to ensure your refactoring will work out well, otherwise you very likely wouldn't be in your current position. If it does they might've been abandoned a long time ago, and it's not even safe to say if the tests are still testing the right thing. If you have a big controller with a good test suite in place, even better. Check if they're testing all aspects of the code about to be refactored.

If not, take this opportunity to write as much tests for it as possible. Test even the simplest features, with as much detail as you can or as the time available allows for. You don't want even those features to break, do you?

I easily end up with 50 new test cases for a bigger action during such a run. Resist the temptation to refactor while you write tests. Mark parts of the code if you get ideas what to do with them, and get back to them in the refactoring phase.

Avoid testing too much functionality at once in a single test case. Keep them small and focused on a single aspect of the code in question. It doesn't have to be tests with just one assertion, but keep it focussed on a specific aspect of the method in question.

Basically it's now or never. This is the chance to improve test coverage, so do it. You'll be happy you invested the time, it will give you a better understanding of the code, and will ensure that the code still works.

It’s a painful process, but it also helps you to really understand what the code does.

Avoid complex fixtures

I don't use a lot of fixtures anymore in my functional tests. They're not just a pain in the ass, they're hard to set up, especially for complex actions, and they kill test performance. Try to use mocks and stubs instead. If you test your action line by line you can easily identify the methods that need to be stubbed or mocked. If you prefer it the scenario way, use something like factory_girl to setup objects for your tests. I'm a fan of stubbing and mocking, but too much of it will clutter your test code. I've been using it heavily for a while, but it tends to become a sign for bad design when you're using too much of it. So I've returned to using scenarios based on the task at hand, even if they hit the database.

If you turn to mocking/stubbing initially, make sure you untangle the potential mess afterwards. Even though the database can make your tests slower, in the end you want to test the whole thing.

You also want to stub out external collaborators, like web services, Amazon's S3 and the like. They don't belong into your controllers anyway, but moving them somewhere else might just open another can of worms (think asynchronous messaging), and introducing that level of complexity is just not what you need right now. Though you might want to consider it eventually.

Move blocks into methods

I'm not speaking of a block in terms of proc and lambda, but in the sense of conditional branching and the like. Longer if/else clauses usually are good candidates for getting code out of a long method into a new one, and you usually can go ahead and do just that. Once you've moved stuff out into methods, it's a lot easier to move them into modules or the model, but only if the blocks depend on parameters you can't or don't want to reach in your model.

Try to avoid the temptation to look for similar code to consolidate in the same or other controllers just yet. Make a note, and wait until you have tests in place for all the affected code. Then start to make the code DRYer by moving it into a place more common for all the classes that require it.

Break out new controllers and new actions

The same rule that applies to adding new actions to controllers also applies to working on existing ones: Having lot of actions in one controller usually means that it's doing more than it's supposed to. More and more actions usually mean there's code that's not really the controller's responsibility, solely speaking in terms of concerns. If the controller responsible for logins also takes care of a user's messages, then it breaks the separation of concerns. Move that stuff out into a new controller.

But if you can, it's also feasible to break out new actions. That's a good option when you have an action that responds differently based on input parameters or depending on the HTTP method, an old Rails classic. It will have the advantage that stuff like error handling will get a lot simpler. Big actions that do different things all at once tend to have a complex setup for catching errors. Several variables are assigned along the process, and at the end there's a long statement that checks if somewhere along the way an error occurred. If you separate the different tasks into smaller actions, you'll end up with much simpler error handling code, since it can focus on one thing, and one thing only.

The same goes for all classes really. Although with a model it's not always easy to break out a new class. But what you can do is break out code into modules and just include them.

Extract filters

Filters are a nice fit for parts of the code where objects are fetched and redirects are sent in case something isn't right, especially since the latter always require that you step out of your action as soon as possible, before doing any further logic. Moving that code out into methods, checking their return code and returning based upon that will make your code look pretty awkward. Filters are also nice to set pre-conditions for an action, pre-fetch data and the like. Whatever will help your controller actions do their thing with less code, and doesn’t fit into the model, try fitting it into a filter.

Try to keep them small though. It's too easy to just break out filters instead of moving code into the model, and it will slightly improve the code for sure. But what you really want is a small and focussed controller with a couple of lines of code in each action, and a few supporting filters around them.

Move code into the model

This is where it gets tricky, but now you can get all that business logic where it belongs. To get code out of the controller and into the model, you have to make sure it doesn't rely on things that's only available in it. params, session, cookies and the flash are the usual candidates here.

But there's always a way to work around that. Oftentimes you'll find code that assigns an error message or something similar to the flash. That kind of stuff is sometimes easier to handle in validations in the model, if it's dealing with error messages. I've seen that a lot, and it's just not the controllers responsibility to do all that work.

If your controller code is heavily dealing with stuff from the params hash, you can usually just hand that over to the model. Given of course that you properly cleaned it up first into a before_filter, or ensured that proper validations are in place.

You’ll usually find lots of constructed finders in controllers. Go after those too. Either use named scopes if you can, or create new finders in the model. It's already a lot easier on the eye when all that hand-crafted finder code is out of the way, and tucked neatly into a model.

Code that checks model objects for errors, validity, etc. belongs into validations or callbacks in your model. Just like any other code that’s none of the controllers’ business. Which basically is to mediate between the view and the model, and to do everything required to get that task done fast and without hassle. So that's the next step. A lot of times you'll find controllers setting arbitrary instance variables based on the state of the model. Rings a bell? Sure, why should the controller store the state of the model? It just should not. That's what the model is for, right?

When you’re done moving code into the model, move the according test code from functional to unit tests. Tests that used to test the business logic from the controllers perspective can now do so from a unit test. That way your functional tests can solely focus on what your web layer does.

Over time you will get an eye for code that just belongs into the model, and code that could be moved into the view, or that needs to stay in the controller. It takes practice, but the more the better. Working with legacy code is oftentimes an opportunity, not a punishment.

Know when to stop

Now that you have a skinny and tested controller, why not just keep going? It’s easy to fall into the refactoring trap. It’s just such a nice feeling of accomplishment. If you look at it that way, you could just keep refactoring your application’s code forever. But who will build the new features? Who will fix the bugs?

Avoid refactoring for the sake of it. Refactoring is an important part of the development life-cycle, but you need to find a balance between your role as a programmer, where you add new features and write tests for them, and the refactoring part.

So I could say “rinse and repeat”, but when you’re done with the controller at hand, leave it be. Get a coffee, and bask in the glorious feeling of just having done your code and the developers to come, a big favor. Unless of course, you have a lot of time. In that case, for the love of code, keep going. But that’s usually a luxury. What you can do instead is plan in time for more refactorings when you’re adding features on a controller that’s another mess. Clean it up first, then get going with the new code. When you see code that could need a refactoring while working on something different make a note (and by note I don't mean TODO, FIXME, and the like, they will get lost in the code, and never be looked at again), and get cracking on it later.

This was just a beginning though. There's still things that surprise me when I work with legacy Rails code, and they want to be dealt with in their own specific ways. As I mentioned earlier, I'm still trying out new things, and I'm going to keep posting about them.

Please, share your experiences on the matter. I'm pretty sure I'm not alone with the pain of oversized controllers.

On a current project we recently switched from Ferret to using Solr as our search engine. That switch was more than necessary, but that's material for a different blog post. Let's just say, the switch was more than worth it, and Solr just rocks our socks off.

The decent choice to turn to was of course acts_as_solr. Unfortunately I soon realized that there was no real active development, except for a couple of forks on the GitHubs. Luke Francl's was the most recent one at that time, so I forked it and started adding my own extensions and wrote a new unit test suite from scratch using Shoulda.

The project seemed to be missing an active maintainer, since Thiago stepped down a few months ago. Talking to Luke and the mailing list, a new plan was made, and without further ado I give you a fresh acts_as_solr mainline. I had weird feelings about picking up development, but I didn't want to let the plugin go to waste in the midst of a million forks. Of course Luke has commit rights to the repository as well.

Since the RubyForge project page is not up-to-date, I'll try to evolve most of the information and documentation on the project around the wiki. An rdoc documentation site will follow.

Drop by the mailing list, there'll surely be someone to answer your questions. I'm open to patches, so if you have something to contribute, feel free to drop me a patch or a pull request.

Tags: rails

It's true, I did write my diploma thesis using Vim. Old-school with LaTeX and C++. When I came to the Mac five years ago I was still using the now pretty much dead Carbon version of MacVim. And well, it just didn't feel right. I'm very comfortable on the command line, but on the Mac I wanted something that integrated well with the rest of that system, that behave like a real Mac application.

Of course, like a lot of people, I found TextMate. Who could resist the cool stuff as seen on the first Rails screencasts? I sure couldn't. I've been using it a good three and a half years now, and I still like it.

But recently the new Cocoa version of MacVim scratched my itch. It was Jamis Buck's blog entry that eventually pushed me over the edge, and had me trying out working with Vim for the last week or so. And holy crap, a lot has happened in the world of Vim since I left it for good. Omni completion, a really good Rails script package, and lots of other cool stuff.

So I gave it a go, and it was like Jamis said, it kinda felt like coming back home. I spent most of my university days with Vim, actually the first years using the old-school Solaris vi. So it basically felt like I never left.

I got pretty fluent with it pretty quickly, and started looking for a nice set of scripts that would fit my workflow. I found

It all felt pretty good in the beginning, especially rails.vim is an amazing package. But after using it for a week it made me realize one thing: That I haven't dived into the Rails bundles deep enough. There's a lot of things in rails.vim that the TextMate bundle also has. What is seriously cool in Vim is the completion, but I just don't use it that much, and it can be frickin slow if you're on a big project.

And that's mainly my main gripe, it all didn't feel very speedy. It took one to two seconds for a normal file to load, what with all the BufRead action going on for Ruby and Rails project files. I didn't mind it that much in the beginning, but it got really annoying. Plus, a lot of the plugins, like NERD tree, or taglist felt kinda bolted on.

So here I am working in TextMate, still loving Vim for it's flexibility and simple effectiveness, promising myself to delve deeper into what the bundles offer. It was a great week, and I'm glad that Vim gets the love it deserves.

One issue that drove me back to Vim was the fact that there's no news on what's happening in TextMate development, and what will be in 2.0. What the week in Vim made me realize were that TextMate could use stuff like split-screen editing, the ability to handle bigger files without hogging memory and CPU, and maybe some real good SCM integration.

My biggest gripe though was that file types didn't stick, switching from Rails to RSpec and Shoulda and back just seemed to confuse TextMate. I was made aware that there actually is a "fix" for that problem, but that just isn't a full solution. It helps right now, but I can only hope that TextMate 2 integrates something like TabMate, just maybe not with modlines but with metadata.

IntelliJ used to be my favorite Java IDE, it was seriously the tool that made the pain of working with the Java enterprise stuff bearable.

On November 1st they released a public preview of their Ruby IDE RubyMine.

Picture 1

It does feel a little shaky still, I couldn't get debugging to work, but it's a good start. I'd love having something like IntelliJ at hand, not necessarily for every day work, but especially for stuff like a good debugging session.

Keep it coming, JetBrains.

If you want to try it out, make sure you have a lot of screen real-estate at hand.

Tags: rails, ruby

At RailsConf Europe, Yehuda Katz showed off a small yet totally useful feature of Merb. A method called run_later that does nothing more than queue the block it receives as an argument to be executed after the request is done.

After the announcement that Rails 2.2 would be thread-safe, and after seeing Nick Sieger's work on a connection pool for ActiveRecord I thought that this feature should be usable with Rails as well.

So without further ado, run_later is now available as a plugin for Rails. It simply fires of a separate thread when your application starts and will work through whatever you put in its queue.

The worker thread will be on a per-process basis due to Ruby's in-process threading. So each Mongrel you start will have its own worker thread. It's not the place to put long running tasks that need some kind of separation or need to run in the order they arrive in the queue. For that, turn to tools like Starling/Workling or ActiveMessaging. Also, it doesn't use a persistent queue, so if you have important messages to deliver, again, turn to more advanced tools.

But for smaller tasks that don't need to run within the context of the request or would potentially block it, run_later is pretty much all you need.

Integrating it in your controller is simple:

The only requirement: Rails 2.2, the current edge version works fine. With any version lower than that especially the behavior of ActiveRecord will be unpredictable. But if you're using Rails 2.2, give it a go, and let me know about your results. Mind you, I haven't put this into production myself, so some stability testing is still on my list.

I'm planning to throw in some kind of scheduling so that you can say something like run_later :in => 10.minutes.

Credit where credit is due: To Ezra for adding the code to Merb, for Yehuda to mention the feature at the BoF session and giving me the idea to port it. Not to forget the effort to make Rails thread-safe in the first place. I merely adapted their code to work with Rails, and threw in some tests as well.

Tags: rails, ruby

A confusing title, I know. But I recently upgraded a rather big project to use Rails 2.1. Everything went pretty smoothly, but one thing bugged me, since it's not really documented anywhere: What happens if you migrate from the old numbered migration scheme to the new one using UTC timestamps?

The new migration system dumps every migration ever run into a new table called schema_migration. That of course includes your old migrations, at least those that exist in db/migrate at the time you first run rake db:migrate on a Rails 2.1 project. It will silently drop the old and trusty schema_info table, and from then on you're good to go to use the new naming scheme for migrations.

So migrating a project to use the new migration scheme is as simple as running rake db:migrate once. Check that the table schema_migrations hasn't been created accidentally though. That will just fail inserting the existing migrations.

There, that was easy.

Tags: rails

RailsConf Europe 2008 is over (and has been for a few days now, I know), so it's time for a recap. In all it was much better than I expected, last year's conference was a bit of a disappointment, so my expectations were low enough to be positively surprised.

Bratwurst on Rails was, if not a huge success, a great event nonetheless. Probably because of the rain not as many people showed up. There werre around 200 guests, but I sure hope they had a good time. In the end most of the cupcakes were gone, and all the bratwursts. So it's all good.

Day 1 started with the tutorial sessions, including one I hosted with Jonathan Weiss on deploying and monitoring Rails applications. Over the course of four hours we gave an introduction on deployment setups, doing the dirty-work with Capistrano, and doing basic monitoring. The slides are up on Slideshare, enjoy!

The day ended with a panel consisting of David Heinemeier Hansson, Jeremy Kemper and Michael Koziarski. They talked pretty much with themselves for the lack of questions from the audience. Someone came up with the question (probably for the millionth time), if Rails is gonna switch to RSpec for testing.

RejectConf sure was a highlight in itself. Good fun, entertaining people and drinks in a pleasant environment. What more could you ask for?

Second day started off with David's keynote, and I gotta say, it was most excellent. He talked about legacy code. Still working on the oldest piece of Rails software he knows his fair share about it. Lots of programmers coming to Rails tend to forget that eventually every new project they work on will turn into legacy code, and needs to be maintained. So David's talk took the more inspirational approach, compared to his previous keynotes.

Quick summing up of the day's sessions:

  • Jon Dahl's talk gave a good introduction on MapReduce, not specific to a certain framework, but to how MapReduce works.

  • Yehuda Katz's talk on jQuery sure filled up the seats quickly. And it actually was a pretty good introduction. Favorite quote (more or less accurately): "Valid markup is when every browser can display it, not when the validator says it's valid."

  • Intellectual Scalability presented an approach on how to scale applications by separating them into different micro-applications running on different servers.

  • Wilson Bilkovich's talk on Rubinius gave some updates on the state of Rubinius, but was probably more interesting for people interested in building compilers (LLVM, if you must know).

  • Jonathan's talk on Security on Rails attracted a big crowd too, and rightly so. Hadn't seen it before, so I can safely say I learned some things as well.

The day ended with Jeremy Kemper's keynote, though I think that it'd have fit better into a normal session. It was a good talk on performance of Rails applications, but it wasn't exactly keynote material.

I attended the BoF sessions on MagLev and Merb, and both were pretty good. One thing I didn't understand is the heat some people gave the guy from GemStone. MagLev is still in its early stages, but I'm so looking forward to giving it a spin once it's ready for the public.

On to day two

  • It started off with with Matt Wood's talk on Genomes on Rails. He's working on an impressive project, involving an even more impressive amount of data.

  • Jay Fields talked about lessons learned from functional testing. Pretty good talk. I can't say it was all news to me, but still a good inspiration for people not yet into testing.

  • Justin Gehtland's talk on Small Things, Loosely Joined and Written Fast sure was one of the best of show. He's an excellent presenter, and introduced using ActiveMessaging and RubyCAS (welcome to my list of things to try out) to keep things loosely coupled.

  • For lack of anything more interesting I attended the talk on "Treading Rails with Ruby Shoes". Let's just say it was a different approach on presenting. And that's that.

  • Tammo Freese flashed the crowd with some serious Ruby meta magic afterwards. Tough stuff, but it still matters to people writing and/or using plugins.

  • I finished off the day with Adam Keys' talk on handling complexity in Rails applications. While nothing new to me (I'd used the things he mentioned in several projects already) it gave a pretty good overview on how to handle some of the complexity in Rails applications.

In all it was a pretty good conference, I met a lot of nice people and had a pretty good time. Sadly it won't be in Berlin next year, but let some other European city be the center of the Europe Rails community for a change.

If you're up for more details on the sessions, Clemens Kofler has excellent coverage.

Tags: rails, railsconf, ruby

The Real Adam on complexity:

"My stance is that all the really simple applications are done. But perhaps there are some ideas, some complexity, we can add to our problem-solving repertoire that let us tackle much larger applications."

He's giving a talk on the topic at RailsConf Europe. It's an interesting topic, and I'll surely be there.

Tags: rails, railsconf

Rails is growing up. Almost three years after it's 1.0 release, Rails is going to be thread-safe. Charles Nutter has the inside scoop on why it's important.

Excellent work, Josh Peek, freshly appointed Rails core member.

Tags: rails

That's what it's been. And who needs to work when there's so many nice projects to work on, eh? Well actually, I did work, but in my free time I also worked on some other things, a new one, and existing projects.

I started wrapping up Macistrano for a first release. It's looking good so far, but I still need some new icons, so that it won't look like you have two instances of CCMenu running in your menu bar. If you're wondering what it is: It's a small desktop frontend written in RubyCocoa (so yes, it's Mac only) to monitor deployments running on Webistrano. Webistrano in turn is an enhanced web frontend for Capistrano. So as you can see, there's lots of meta involved here. I basically built a tool to access a tool that runs a tool. Neat stuff, eh? If your deployment is finished in Webistrano, Macistrano will notify you through the convenience of your desktop. You can also fire off deployments with it.

Speaking of Webistrano, I had good fun working on it too. Mainly some stuff that I wanted to see in it, like code preview for recipes, syntax check, versioning. But something that really scratched my itch was import for existing Capfiles, so I just had to try implementing it. As Jonathan will no doubt confirm, it was one of the first questions that pop up, when you tell people about Webistrano: Can it import my Capfile? Fret no more, it might just do that in the near future. Nice experience, because your definitely have to have a look at the Capistrano source to find out, how it actually stores the configuration internally.

Then there's ActiveMessaging, a small library I have a certain knack for. I wanted to see support for storing messages in case the message broker is down. JMS has something similar, so why can't ActiveMessaging? I built the initial code in February, but didn't get around to actually finishing it until recently. What it does is save your message through ActiveRecord if there's an error that indicates your broker is down. That way your application most likely won't be harmed if your messaging infrastructure has a problem. To recover them you can simply run a daemon that will regularly poll the storage and try to deliver the messages. The code didn't make it to the trunk of ActiveMessaging yet, but you can have a go at it on GitHub.

I also read "The Ruby Programming Language", a highly recommended book. A detailed review will follow. But first I'm off to my well-deserved honeymoon.

After more than two days of removing deprecation warnings, adding plugins, fixing some custom additions, going through the whole application, it's finally done. We're running Rails 2.0. Nothing more gratifying than seeing this, well except for the application running without problems:

Picture 1

There were some minor annoyances, but in all it was straight-forward work. One thing was that actsasferret 0.4.0 does not work with Rails 2.0.2, but the upgrade to 0.4.3 doesn't go without any pain either. In 0.4.1 index versioning was introduced which will happily start indexing your data when you first access the index.

Be sure to have the exception notifier plugin ready, that will help you find some of the bugs you might have overlooked.

Rails 2.1, we're ready for you!

This is the bundle that'll make TextMate almost as good as warm apple pie. RubyAMP comes along with a few additions to Ruby coding in TextMate, but boy do they rock.

Some of its features:

  • Grep for Method
  • Grep for Class/Module, both method and class/module grep use the word under the cursor
  • A much faster (compared to "Find in Project") "Grep in Project"
  • Code completion with data from all open files
  • An intelligent "Go to file" which will try to find something matching the word under the cursor
  • Run RSpec in the debugger

Here's code completion in action:

Picture 2

On the website you'll find a video showing most of the features. Pretty neato. Ruby and Rails development on the Mac just got even sweeter.

Via Samurai Coder.

Lots of open tabs to be swept. Brace yourselves.


Can you tell I'm looking into Git right now?



Mac OS X

  • Secrets. An ever-growing list of hidden preferences. Comes with a handy-dandy preference pane.
Tags: links, macosx, rails, ruby
Tags: links, rails, ruby

The title totally overemphasizes the topic, but here we go. By default ActiveMessaging will process all your queues in one thread. All messages from all queues will be processes sequentially. This isn't always what you want. Especially in scenarios where you have both long-running tasks being kicked off through messages, and rather short-lived tasks that you just fire and forget.

ActiveMessaging has a rather simple and not-so-obvious way of dealing with that: processor groups. There's some documentation on them, but it doesn't bring out the real beauty of them.

Basically you split your processors in groups, how finely grained is up to you. A simple way would be to just separate long-running from short-lived tasks. You just have to define these in config/messaging.rb:

ActiveMessaging::Gateway.define do |s|
  s.destination :index_update, '/queue/IndexUpdate'
  s.destination :giant_batch_job, '/queue/GiantBatchJob'
  s.processor_group :short, :index_update_processor
  s.processor_group :long, :giant_batch_job_processor

Now that you have these, how do you get them to run in different threads? If you just use script/poller start, it will continue to work through all messages from all queues. You need to start each processor group individually:

$ script/poller start ---- process-group=short
$ script/poller start ---- process-group=long

Keep in mind though that you can't stop just one poller for one particular processor group. Running script/poller stop will tear them all down. Which comes in handy during deployments. That way you only have to ensure that all your process groups are started during the deployment, but not about stopping every one of them.

ActiveMessaging will run each group in a separate thread which all are monitored by the poller_monitor. The latter will only be started once, courtesy of the daemons package.

I've been playing around with ActiveMessaging recently. Well, actually more than that. I integrated it into a project for asynchronous processing. It's a pretty neat plugin. We're using StompServer as a message broker, and therefore the Stomp protocol to publish and poll the messages.

Now Stomp is a pretty simple protocol and breaks down when you're trying to deliver "complex" data structures like hashes, arrays or *gasp* objects. That's not a bad thing per se, since we can serialize them with YAML. Of course you could just always do that by hand before publishing a message, but let's face it, that's just tedious.

The author of ActiveMessaging recently added support for filters. They can be run after publishing a message and/or before processing it on the polling side. I hear it clicking on your end, why not use filters to do the serializing work for us? Right on!

Here's a simple filter to serialize the message when it's sent:

class SerializeToYamlFilter < ActiveMessaging::Filter
  attr_accessor :options

  def initialize(options={})
    @options = options

  def process(message, routing)
    if message.body.respond_to?(:to_yaml)
      message.body = message.body.to_yaml
      message.body = YAML::dump(message.body)

It uses the to_yaml method mixed in by Rails, if it's available. Otherwise it just dumps the object with the YAML::dump method.

The receiving end is even easier.

class DeserializeYamlFilter < ActiveMessaging::Filter
  attr_accessor :options

  def initialize(options={})
    @options = options

  def process(message, routing)
    message.body = YAML::load(message.body) rescue message.body

The filter respects potential deserializing errors and just returns the message body in that case. Otherwise it just loads the objects from the message body. And that's the whole story.

Now you need to configure it in config/messaging.rb and you're good to go:

ActiveMessaging::Gateway.define do |s|
  s.filter :deserialize_yaml_filter, :direction => :incoming
  s.filter :serialize_to_yaml_filter, :direction => :outgoing

The benefit? This way you can send more complex data structures (as opposed to just strings) through the broker:

publish :my_queue, :action => 'do_stuff', :with => 'User', :id => 1

But remember to keep it simple. Don't try to stuff large objects through there. Sending over the user itself is very likely not a good idea, even more so when it's an ActiveRecord object.

More to come on a13g and Stomp.

Another week, another sweep.

Tags: links, rails, ruby
Tags: java, links, rails, ruby
Tags: links, rails, ruby

The book market is being swamped with new books. It seems like every day I discover a new announcement for an upcoming book on Ruby or Rails. Let's see what's currently in stock, and what's waiting for us next year.

The first ones to start are, obviously, the Pragmatic Programmers.

Apparently some of the authors of RSpec are also writing a book about Behavior-Driven Development for them.

Apress quickly became big on the Ruby/Rails bandwagon, and speaking of RSpec, they also have a book on it due in February, though it's not officially in their catalogue yet.

Just recently they put out several books:

  • Pro Active Record. I'm still wondering how it's possible to write an entire book on Active Record, but apparently it's possible.

  • Practical JRuby on Rails Web 2.0 Projects. I don't fancy the title, but I ordered this book anyway, since it's pretty much the only one on the topic of JRuby.

  • Practical Rails Projects. Shows different aspects of Rails by using them in very different project. A nice approach compared to just developing one or two applications over the course of a book.

Upcoming books from Apress:

O'Reilly already put out their share of Ruby and Rails books, but right now the queue looks rather slim.

  • Advanced Rails. Seems to cover several topics of Rails that are, well, advanced, covering topics like Meta Programming, REST, developing plug-ins, etc.

  • The Ruby Programming Language. Mats itself is writing this one together with David Flanagan (who wrote a lot of books on JavaScript). I'm not sure if it can replace the PickAxe as the most valuable Ruby book for me, but that might just be for strictly melancholic reasons, what with it being my first Ruby book.

Manning has only put out one book on Rails yet, the excellent "Ruby for Rails" by David Black. But they have some interesting stuff in the queue for 2008.

  • Ruby in Practice. This could become the mother of books on advanced Ruby topics. The topics include testing, integration, indexing, asychronous messaging and much more.

  • Flexible Rails. I can't say much on that topic, but maybe this book will change that. Flex and Rails. Nuff said.

Addison-Wesley recently released kind of the companion book to "The Ruby Way", a most excellent book.

  • The Rails Way. Just been released, covers a lot of Rails grounds, from basic to advanced. Should make for a nice desktop reference.

  • Design Patterns in Ruby. Though I have mixed feelings about design patterns and their overuse in the Java world, it should be interesting to read about how they can be useful in the Ruby world.

There, lots of new stuff to read for the new year. I'm sure there's more to come, as it naturally happens with a technology on the rise. The downside is that the quality of the books becomes more and more average over time, but there's a lot of talented people writing on these, so hopefully the Ruby/Rails book market will see more and more great books coming out in the near future.

Tags: books, rails, ruby

Mocking is a great part of RSpec, and from the documentation it looks insanely easy. What had me frustrated on a current project is the fact that the mocks and stubs wouldn't always do what I'd expect them to do. No errors when methods weren't invoked, and, the worst part, mocks wouldn't be cleaned up between examples which resulted in rather weird errors. They only occurred when run as a whole with rake spec but not when I ran the specs through TextMate.

I was going insane, because noone on the mailing list seemed to have any problems, same for friends working with RSpec. Then I had another look at the RSpec configuration.

Turns out, the reason for all of this is that Mocha was used for mocking. Switching the configuration back to use RSpec's internal mocking implementation, everything worked like a charme from then on.

So what you want to have in your SpecHelper isn't this:

Spec::Runner.configure do |config|
  config.mock_with :mocha

but rather

Spec::Runner.configure do |config|
  config.mock_with :rspec

or no mention at all of mock_with which will result in the default implementation being used which is, you guessed it, RSpec's own.

Friday's tab sweep for a little weekend reading.

Tags: links, rails, ruby

When using CruiseControl.rb for continuous integration, and RSpec for testing, the defaults of CruiseControl.rb don't play that nice with RSpec. However, that can be remedied pretty simply.

By default CruiseControl.rb runs its own set of Rake task, which invoke a couple of Rails' tasks, including db:test:purge, db:migrate and test. You do want the first ones, but you'd rather have CruiseControl.rb run your specs instead of your the (most likely non-existing) Test::Unit tests.

Before running its default tasks it checks whether you have a specific task configured in your project settings, and if you have a task called cruise in your project's Rakefiles. You can use both, but I just declared a task cruise in the Rakefile, and I was good to go.

That task can to pretty much the same as the original CruiseControl.rb implementation, and even be shorter since it can be tailored for your project, but invokes spec instead of test. One caveat is to set the correct environment before running db:migrate. I split out a tiny prepare task which does just that, and can do a couple of other things, if necessary, like copying a testing version of your database.yml.

desc "Task to do some preparations for CruiseControl"
task :prepare do
  RAILS_ENV = 'test'

desc "Task for CruiseControl.rb"
task :cruise => [:prepare, "db:migrate", "spec"] do

Simple like that. The task spec will automatically invoke db:test:purge, so you don't need that.

Today's NetNewsWire tab sweep:

  • DataMapper - Competition for ActiveRecord - Not available at the moment, cached on the Googles.
    Looks like a nice alternative to ActiveRecord, especially for standalone environments without Rails. Being able to put queries, especially simpler ones, in Ruby code is quite appealing.

  • The Great Ruby Shoot-Out
    Antonio Cangiano compared the current Ruby implementations. JRuby, Ruby 1.9 and Rubinius look awesome, at least by the numbers.

  • Upcoming Changes to the JavaScript Language
    I'm still not sure if I'm gonna like what's coming. JavaScript is on the way to turning into a full-blown and statically typed object-oriented language, but with all its pros and cons. It looks a lot like C++, and if that's no a little bit scary, I don't know what is.

  • Google Chart API
    Now that's a really useful API. Put in your data via HTTP request and get back chart images. Awesome.

  • SVNMate
    A TextMate plugin to integrate Subversion, taking the integration further than the Subversion bundle.

  • Rak
    grep in Ruby, ignores .svn and CVS directories. Accepts regular expression in Ruby syntax, and can execute Ruby code on the results.

  • The Evolution of Java
    Right on.

On a side node, Rails 2.0 has been released. I recommend checking out the PeepCode book on Rails 2.0 to check what's new and what's old.

Update: Ryan Daigle (author of aforementioned book) also has a nice collection on a lot of the changes and new features in Rails 2.0.

  • QuickLook and TextMate, sitting in a tree. Makes QuickLook even better. It integrates the TextMate syntax highlighting into QuickLook, and integrates QuickLook into TextMate. Looks pretty neat. TextMate in QuickLook

  • Ruby Tool Survey
    It's official (as far as this survey goes): TextMate is the number one development tool for Ruby and Rails. That's what I tried to tell the students during the lecture I gave last weekend, but the die-hard Linux fans wouldn't believe me ;)

  • Nginx and Mongrel for Rails
    Nice tutorial on how to set up Nginx for Rails deployment. I'm using it on one of my current projects and it's pretty neat. RailsJitsu is well worth keeping an eye on.

  • Scripting the Leopard Termina
    Tired of having to open four terminal session each time you open Terminal to work on your Rails applications? Get some nice hints on how to script Terminal with AppleScript to ease the pain on your fingers.

  • Inject Ruby into a Running OS X Application
    Just cool. Nuff said.

Tags: links, rails, ruby

For a recent project I had the pleasure to work with Paypal, especially with the Instant Payment Notification API. I haven't heard a lot of things before I tried to marry it with Rails, but what I'd heard made me assume it wouldn't be a piece of cake. And I was right.

I'd love to share some code with you, but Vasillis Dimos beat me to it. He wrote two posts on Paypal IPN and Rails, one dealing with the basics and the other about mocking IPN, which you really need to do to test your code. Really.

Personally I did the testing a little differently, since all my payment handling logic was in the model. I didn't use ActiveMerchant either, but just the Paypal gem. But in general things are similar. Outside of the US and the UK you're pretty much out of choices for payments, since there's no Web Payments Pro available here, so IPN is (sadly) the way to go. It's a real PITA and here's why:

  • Paypal needs to reach your development machine from the outside. For testing this is not an issue of course, but when you need to do testing with the Paypal sandbox (which is painfully slow) and, god forbid, the real Paypal, there's no way around that.
  • The payment flow is unnatural. You have to handle the payment outside of the user's page flow. You have to rely solely on the stuff you get from Paypal, no session, no cookie, no user. It takes a lot of care to handle all that and there still might be a hole in your code that could be exploited.
  • IPNs might come in late, sometimes only after the user already got back to your site. Now you want to present him with a nice success message, but that's not gonna happen then. That's a rare case though. The IPN come in slower from the sandbox, that's for sure. It's up to you how to handle that. You can act in the favor of the user, or you can just make him wait till everything fell into place.
  • In rare cases you won't get an IPN from Paypal, for whatever reason. I've seen this happen. Be prepared to create the successful payment by hand or have something like a small GUI at hand to do it.
  • For subscriptions six different notification types need to be handled. And their even spread out over two different fields in the notification.

Some advice on how to get it right: * Log everything. Store the IPNs in the database, in the log files, wherever. Just log them. Their your proof of things that happened. Just storing them with their raw post data should do while leaving the most important fields separately in different columns. * Use mocks. It's not hard. But it's totally worth it. When you want to test all events that Paypal might send you, which is a lot for subscriptions, it's a painful development cycle. And some events aren't even fully testable by hand. * Decide on strategy to handle fraud. While your IPN URL is not really public (nothing should link here, and it's hopefully transmitted to Paypal encrypted) it's not exactly safe to just accept everything. * Don't return errors in your IPN handler. Paypal will try it again. * Store a pending payment and make it a full one when the corresponding IPN arrives.

All that said, it was an experience, and while not always pleasant, at least I learned something. But Paypal is far from being a pleasant way to handle payments, if you want to make it secure and protect your the integrity of your application and prevent fraudulent users from abusing your services, all of which should be your primary interests.

David A. Black of Ruby Power and Light will be giving a four-day training in Berlin from November 19th to 22th 2007. The training targets intermediate Rails developers who already worked with Rails and are familiar with the basics. Topics include ActiveRecord (attribute handling, associations, optimisations), routing, REST, testing and deployment.

You can find the full details (in German) on the official website. Seating is limited, so hurry!

Tags: rails, ruby

Together with Benjamin Krause I wrote an article on the Ferret and acts_as_ferret for the German IT magazine iX which was published today. The article is part of a Ruby on Rails special which also includes articles on the current state of Rails and deployment with Capistrano, both written by Ralf Wirdemann and Thomas Baustert. Go grab it while it's hot (and available).

In other news, everyone's favorite freelancing site FreelanceSwitch recently started a podcast called "Freelance Radio". I highly recommend checking it out.

Tags: rails, ruby

First up: RailsConf Europe 2008 will be in Berlin again. Woot!

As I wrote yesterday, Marcel Molina and Michael Michael Koziarski did a little Best Practices session for a starters. Other than that, day two was only okay.

Ola Bini repeated in his JRuby talk pretty much what Charles Nutter and Tom Enebo said on the first day, plus some shameless ThoughtWorks pluck.

I did enjoy the talk on Selenium by Till Vollmer. It's been on my list for a while, and it does look quite interesting. The questions that pop up in my head as a friend of continuous integration is of course how to automate this. But I'll just have to read up on that.

Ben Nolan (creator of Behaviour.js) showed some neat tricks using functional programming with JavaScript. He brought up some ideas and showed code, which I very much appreciated. Nothing new for the JavaScript cracks really, but still interesting.

Jay Fields talked about the presenter pattern in Rails. I bet a lot of people thought after the talk: wtf? To sum up his findings on the presenter pattern in Rails were rather negative and probably not what a lot of people expected. I found his talk to be a change to the others. It's not always the success stories that make you push hard, but also down-falls, even if they're small ones. He put up all the details in his blog. Definitely worth checking out.

In all I would've wished for more detail in the presentations. A lot of the presenters spent too much time introducing things, presenting theory, and so on. More code please, people! When people come to the RailsConf I take it for granted they know Rails enough to get down and dirty immediately.

As DHH wrote on his blog I too was quite impressed by the engagement of Sun in Ruby and Rails. Craig McCanahan (of Struts fame) talked about it and said he can't imagine going back to Java for web development after having worked with Rails. Amen to that.

I got some nice ideas and things to look into out of it, but I have hoped for more. But still I'm looking forward to next year.

This morning, on day two, Marcel Molina and Michael Koziarski did a little Best Practices session, a welcome change to the keynotes and sessions. It was very code-oriented. I did even take something out of it I didn't know before. Though I wish it would've gone into a little bit more detail (which I actually wish for a lot of the other sessions as well, but more on this in a later post), it was something that you could relate to on a practical level.

I took some notes, without code though, and here they are:

  • Keep the controllers skinny, keep logic that’s on the model's low level out of the controller
  • All that logic in the model makes it the fat model
  • The controller should not deal with that logic, because it’s a different layer of abstraction
  • Rough guide: 6 to 7 actions per controller, 6 to 7 lines per action
  • Use association proxy methods. Add custom finders for associations to keep the association logic in the model and to represent the business logic more clearly
  • Use explicit and short validation callbacks (e.g. validate :make_sure_something_is_as_it_should_be) instead of just long validate methods. It’s easier to read and understand
  • with_scope can make code harder to read and is (apparently) used in situations where it isn’t necessary. It can be used to fake associations through proxies, e.g. to find objects that aren’t associated with an object through the database, but through some conditions, e.g. a smart group or a smart folder

Short, but sweet.

Day one of the RailsConf Europe is over (for me anyway), and so here's my summary of what I've seen and heard today.

It all really started yesterday with Dave Thomas' keynote on "The Art of Rails". The talk was inspiring. It wasn't really any new stuff, basically a nice speech with visuals about what the Pragmatic Programmers have already written about. The comparison to art sounds far-stretched for a lot of people, and it might even be. Still, there's a lot to learn from art that can be applied to software development. Casper Fabricius published a nice summary.

The morning keynote by David Heinemeier Hansson was okay. It wasn't great. It pretty much summed up all the neat new features of Rails 2.0. There's another nice summary over at the blog of Casper Fabricius.

My sessions schedule started out with Benjamin Krause's "Caching in Multi-Language Environments." He actually patched the REST-routing in Rails to support the content language as a parameter for a resource URI, e.g. /movies/231.html.de. Neat stuff. He also implemented a language-based fragment cache using memcached. Both will be available later this week on his blog.

Next up was Dr. Nic's talk on meta-programming with Ruby and Rails. My word, I love his Australian accent. His talk was highly entertaining, I was laughing a lot. But it was also inspiring. He's very encouraging about trying out the meta-programming features of Ruby and doing some weird, funny and useful stuff with it. He already put up his slides for your viewing pleasure.

The afternoon was filled with the wondrous joys of JRuby and Rubinius, held by their respective maintainers Charles Nutter, Thomas E Enebo and Evan Phoenix on both of which I'm hooked now. Especially Rubinius impressed me a lot.

Roy Fielding's talk on REST was something I was really looking forward too, but it turned out to be more of a summary of his dissertation. The part on REST was good, but he spent an awful lot of time telling history and the theories behind REST.

The smaller diamond-sponsor keynotes by Jonathan Siegel of ELC Tech and Craig McClanahan were short, but pretty good I'd say.

In all, the day was pretty good, and I'm looking forward to tomorrow.

I can safely say that "Bratwurst On Rails" was a success. A lot of people showed up at the Kalkscheune, ate Bratwurst and had a good time.

Some statistics: - ca. 400 guests - 800 Bratwursts - 140 chicken sausages - 125 vegetarian sausages - 1100 bread rolls - 150 cupcakes (courtesy of Cupcake Berlin) - 100 brownies (courtesy of Misses & Marbles)

Thank you to all the people who helped organise and run the event, and thanks to our sponsors. Without you, the event wouldn't have been possible.

The first pictures are showing up on Flickr, so keep an eye on the "bratwurst on rails" tag. Here are some of the photos my girlfriend snapped yesterday.

Bratwurst On Rails 2007 Bratwurst On Rails 2007Bratwurst On Rails 2007 Bratwurst On Rails 2007Bratwurst On Rails 2007 Bratwurst On Rails 2007Bratwurst On Rails Bratwurst On Rails 2007

I'm still a little bit exhausted, but it was all worth it.

The RSpec and Behaviour-Driven Development edition: * Behaviour-Driven Development. An excellent introduction to RSpec and BDD by Bruce Tate.

Tags: links, rails, rspec, ruby

For Bratwurst on Rails I implemented a small application to allow for easy signup. We could've used something like upcoming or wevent, but first we wanted to give people an opportunity to tell us what they like to eat and second, we needed to have room for our sponsors.

As you might imagine the application doesn't do a lot, it comes along with a flashy design, two simple controllers and one model. Nonetheless I wouldn't have felt comfortable to put the application out in the open without some testing. While I'm sure I didn't test every aspect of it, it tests enough of the functionality to ensure it really works.

Bratwurst Tests

Testing is the part of Rails that has really grown on me, and that drew me to Rails in the first place. The integration of the different types of tests, the fixtures, the ease of writing tests, all that should be enough reason to write tests even for the smallest projects.

If you're just getting started with Rails or haven't looked into testing yet, I suggest you do so soon. It can save a lot of trouble. I don't want to say time, because in the beginning that might not be true. It takes a while to get into testing and to get so comfortable with it that it's only natural to start writing a test for your controllers before you even think about opening the browser. But in the long run it's definitely worth it.

The funky Growl notification is courtesy of autotest, by the way. A shame it took me so long, but I just started using it, and right now I can't remember how it was like without it. Notification icons and notification code can be found on the blog of internaut

Tags: rails, testing

Several friends tried out Ruby and Rails over the last months. Apart from the fact that most of them like it, but have to get used to the different syntax, there's one question that popped up several times, and that I've already discussed with several long-time Rails users: What IDE are you using?

The answer I give them is always: I use TextMate. I know, I know, it's not an IDE you're saying. I'm well aware of that and I didn't imply it would be. That statement just implies that I don't feel the need to use one.

I'm aware that NetBeans seems to be the king of the hill right now, when it comes to Ruby and Rails support for a Java-style IDE. The latter is what bothers me about it though. I don't need all the fancy assistants, the dialogs to generate code, and I can live without the code completion. I know it's something that's missing when you're starting with Rails and come from a code-completed background, but the conventional approach makes it easier to get used to the way you deal with the framework. That's my experience at least.

The weird thing about that? When I work with Java that's the stuff I use all that stuff. A powerful IDE takes the pain out of Java. And that's the reason why I don't fancy one for Rails development. There is no pain. If there is one, it's very different from the pain of Java development, and it's not the IDE that could help me then. I work fluently with TextMate and the command line, so I have no urge to from typing to clicking to get things done.

I love seeing how the community is pushing the Java IDEs to be usable for Rails development, but right now it's just not for me.

Have a RESTful weekend with some REST-related reading, some of it already a little older, but it's gotten more important to me recently: * Refactoring DayTrader to REST. An article on refactoring an existing application to use REST.

Tags: links, macosx, rails

Yep. It's sold out. Compared to the RailsConf in Portland it took a while, but now you can expect to meet something around 800 people there. I'm really looking forward to it, as well as Bratwurst on Rails which will be doing pretty good as well regarding participants.

Tags: rails

Don''t be tempted to overwrite method_missing in an ActiveRecord-based model class. It will open a can of worms that's hard to close without removing your custom version again.

A lot of stuff in ActiveRecord is based on it, for example all the beautiful finder methods like find_all_by_this_field_and_that_field or simply setting and getting of attribute values. While associations get their own methods, the plain attributes are routed through method_missing. So @user.name and @user.name = "Average Joe" all go through it. You can try that out by overwriting method_missing, strictly for educational purposes of course, and only if you promise to remove it afterwards.

alias :original_method_missing :method_missing
def method_missing(method_name, *args)
  puts method_name
  original_method_missing method_name, args

You'd think that this code shouldn't break anything. I tried it, and the validations stopped working. Since there's a lo-hot going on in ActiveRecord, I haven't dug in yet to have a look why that's the case, again strictly for educational purposes. But I'm curious for sure.

If you want to bring dynamic code into your classes, for example generated methods, you're better off generating the code at runtime, just like ActiveRecord does it for associations.

Tags: rails, ruby

It's been a week full of Rails joy, and a little pain as well, but that's not to looked for in Rails itself, but just some code.

  • Been working with attachment_fu this week. Basically tried out its S3 storage capabilities when I switched from a custom implementation. Pretty neato. I'm starting to dig S3 itself more and more. Mike Clark wrote a nice tutorial on the subject.

  • In his newest tutorial on developerWorks, Bruce Tate writes about using RSpec for behaviour-driven testing. Good stuff, I'm looking forward to the tutorial at RailsConf Europe about Behavious-Driven Development.

  • Chris Wanstrath introduces Ambition, an ActiveRecord extension that makes finding objects more Ruby-like. Being able to write

    User.select { |u| u.email =~ /chris/ }.first

    instead of

    User.find(:first, :conditions => ["email = chris"])`  

    feels at least a little bit more Ruby-ish. Still pretty new, but it looks promising. The full, glorious joy of features can be read in the README.

  • Finally, the Softies on Rails talk about what you always knew deep in your ruby-red heart: Hashes are Cool!

Tags: links, rails, ruby

During the last meeting of the Ruby User Group Berlin, Jonathan Weiss presented Webistrano, a web application for easy deployment based on Capistrano. It was pretty impressive, and now he's released it to the public. On the site you'll find some screencasts explaining the features and the user interface, and the download of course.

What I find neat about it, are the multi-stage capabilities and the multi-user support. It's easy to manage several projects and hosts. As you'd expect, it's a Ruby on Rails application and comes pre-packed with all the required goodies.

Nice work Jonathan!

Tags: rails, ruby
Tags: links, rails, ruby

Due to some developers not so keen about running the tests I got back to my trusty friend Continuous Integration for a project I'm currently working on. Being a big fan of CruiseControl I looked for similar solutions for the Rails and Ruby market. There are several tools you can use, and they have several ups and downs.

  • The Slim Solution: Continuous Builder
    A small application disguised as a Rails plugin which you can hook into your Subversion post-commit. It's part of the Rails repository and, though it hasn't been cared for in a while, still fulfils its purpose. The commit hook runs checks out the source and runs the tests. Should one or (gasp) several of them fail, it will bug you with a nice email pointing out the abounding problems. It's the simplest solution, but it needs a separate environment in your Rails application. Ryan Daigle suggests to use a separate build environment to keep the build away from the rest of your application. This sort of repelled me from Continuous Builder since it also needs a database to work this way.

  • The Hell-Hound: Cerberus
    It's simple and easy to use, it comes as a RubyGem and is up and running in no time. Once installed, projects are added through the cerberus command.
    cerberus add http://some.server/with/my/svn/trunk

    By default, it uses the last part of your Subversion URL to create a name for your project, so you might want to add the option APPLICATION_NAME=<my_project> to specify a nicer name than just trunk.

    When you're done you can easily build your project with cerberus build <my_project>. Cerberus will check for changes in your repository and only do its magic when there are changes. It supports notification through email (duh), Jabber, RSS, CampFire and IRC. It can be run as a commit hook or as a cron job.

  • The Classic: CruiseControl.rb
    Of course I didn't discover it until Cerberus was up and running. CruiseControl.rb is the natural evolution that eventually had to come after CruiseControl, the bigger brother of the Java world. Both are products of ThoughtWorks.

    CruiseControl.rb is, of course, a Rails application. It doesn't need a database though. It's pretty easy to use. You just unpack it, add a project and start the required processes:

    cruise add my_project http://some.server/with/my/svn/trunk
    cruise start

    That's about it. After that you can point your browser to the dashboard application and enjoy the magic of freshly made builds.

    Downsides include support for a wider variety of version control systems. For now, only Subversion is on the list.

All of the above are pretty neat and depending on your continuous integration needs, can be recommended. If you need a fancy web application, there's no way around CruiseControl.rb. Its clear advantage is also that it polls the repository without needing cron or a commit hook. Otherwise I'd recommend Cerberus over Continuous Builder because it's a little bit more flexible and offers more notification options. What I realised is that even in a Rails project where the tests are mainly the proof that code hasn't been broken with a change, it's necessary to check this automatically. No more "But it works on my machine" and that warm, fuzzy feeling of not getting email after you checked in.

Tags: rails, ruby

After playing around with AP4R, the new kid on the asynchronous Ruby block, for a little while, I held a small presentation at last week's meeting of the Ruby User Group Berlin. While it is more of an introduction than a field report I put up the slides as PDF anyway.

One note on the load balancing though. That's the issue that brought up some questions I couldn't answer, so I had another look at the documentation (which is still rather sparse, by the way.

AP4R knows two kinds of balancing. The first one is the distribution of dispatching tasks over a number of threads. That way the messages in the server's queue can be handled in parallel. Since the threads only scale on one host this doesn't allow for load balancing on several hosts.

For this AP4R has an experimental feature called carriers. These allow for the AP4R server to redistribute the messages to a bunch of servers exclusively dealing with the messages. It should be added though that these carriers use polling to fetch new messages from the master server's queue. This has the clear advantage that new carriers can be added without changing any configuration on the other servers. Carriers aren't considered stable yet, but they point in the right direction.

As for the client, let's say a Rails application, it can only handle exactly one AP4R server to transmit its messages to. So if you're balancing your application's load over several servers, you can either send all the messages to one server or have each application deliver them to its own AP4R server. The downside of this is that if one server fails, one or all of your server can't deliver asynchronous messages. So it's probably best to always rely on store-and-forward to ensure that your messages won't get lost.

For some further information I'd recommend checking out the website, the "Getting Started" document, where they show how to set up the example application, and a large set of slides from a presentation at this year's RubyKaigi.

I'm looking forward to seeing what's next for AP4R. It's a promising start.

Tags: rails, ruby

For a current gig I had to set up a Java web project from scratch. The set-up isn't that unusual: * Spring 1.2 (yes, it's rather old, but the project uses a portlet framework which used to work with Spring 1.1, and I don't want to make the big jump just yet, the upgrade to Spring 1.2 went just fine though) * Hibernate 3.2 (hence the upgrade to Spring 1.2) * Tomcat 4.1.38 (again, the framework, and it's a very stable version)

What amazed me was the time I needed to set up the project and to get the first code running, including access to the database, mapping a simple class, deployment, deployment descriptors, the whole bunch. Although I used the skeleton of an existing project in a tidied-up version it took me a day until I had all the pieces together, including a 27 MB download just to get the MySQL JDBC connector (whose JAR file is smaller than 500 kB by the way). Granted, it's been a while since I set up my last project, but half a year isn't that long, so I'm not a total newbie at these things.

Here are the basic steps: * Set up a project * Collect the libraries - Downloads from several locations * Create a build file - I had one already from the existing project which just needed some cleaning up. It's an Ant file. "You could just use Maven" i hear you say. Yes I could, but therefore I'd have to get to know Maven which would cost even more time. * Create a Spring configuration - Again I could base this on the existing one, Hibernate needed some polishing * Write mapping files * Create deployment descriptors * Start writing code * Create the database * Deploy the project * Run Tomcat

Now, compare that to this: * gem install rails --include-depdencies * rails my_app * Create the database * Edit database.yml * Write a migration * Write code * Run script/server - No deployment descriptors needed

I didn't have to leave the command line once to do all these things. And if I need new libraries, gem install or script/plugin install are my friends.

It took me almost a day to get everything up and running in the Java project. It usually takes me five minutes to crank out the first features with Rails.

I'm left wondering why things ever got so complicated in the Java world. Things get even worse and more complicated if I'd want to set up a test environment. Are these things so much part of Java web development now that nobody's bothered by complicated set-ups or do people have their own project skeletons which include everything needed similar to the skeleton that Rails generates?

Tags: java, rails

Registration for RailsConf Europe in Berlin opened its doors yesterday. So if you want to have one more reason to come to Berlin, head over there already.

The Ruby User Group Berlin is having a party the evening before the conference, on September 16th. So if you're in town for the conference, why not drop by, say hello, meet fellow Rails friends and get free food. The event's name (and theme) is Bratwurst On Rails, and the registration is open as well. The location is yet to be announced, but that shouldn't prevent you from signing up, right?

The layout of that application has been done by lovely Joerdis Anderson, whose work you can witness over at her site Be A Good Girl.

Tags: rails, ruby

If that's not exciting, then I don't know what is. Original Ruby finally has got competition. I tip my hat to the team that developed JRuby in such a short time frame and now fully conforms with Ruby 1.8.x. Though I'm not keen to run JRuby in Glassfish, it's nice to have the option to integrate a Rails application with J2EE services. Think of that EAI buzzword that came up a few years ago.

Being able to integrate EJB, JMS, or just every kind of Java backend with a (I'm tempted to put some buzzwords in here, think lightweight, agile, full-stack, etc., but I grew pretty tired of these) web framework like Rails is always a nice option to have. Now there's no excuse when Rails is put on the table in a Java/J2EE shop.

Charles Nutter's announcement can be found in his blog. Head over to the JRuby web site and give it a go. I know I will.

Tags: rails, ruby

I held a small presentation about BackgrounDRb yesterday at the meeting of the Ruby User Group Berlin. It wasn't a very big presentation, since actually, there's not a lot to say about BackgrounDRb. But if you want to check it out, you can get the slides and the little sample application.

It's nice to see the user group grow. Yesterday there were about 35 guests, the month before almost 50. If you live in Berlin or in the area, it's definitely worth getting to know the people there. And there's a good chance that you can get new gigs there as well.

In other news, my newest article got published on FreelanceSwitch.

Tags: rails, ruby

A chroot environment seems to be rare these days. Everything is virtualized, load-balanced and what have you. I recently found myself trying to deploy into a chroot’ed Lighttpd environment with Capistrano and immediately ran over several pitfalls. The biggest problem is that Capistrano uses absolute links for directories like current and the links to the log directory.

A base directory for your app could be /var/www/rails and your Lighttpd runs in a chroot environment in /srv/lighty. In the application directory Capistrano creates the directories releases and shared. After a successful deployment it also creates a symbolic link named current pointing to the latest release, and several links to directories in system.

In this scenario the link current would point to the directory /srv/lighty/var/www/rails/releases/20070506135654. Now, since Lighttpd doesn’t know the directory structure above /srv/lighty that’s a bit of a problem and it won’t find the path when you point it to the dispatcher in the directory current. This is true if you launch your Rails application through FastCGI. In a Mongrel scenario it would pretty much result in the same problems. Additionally, your Rails application won’t find its log and other directories (if you’re up for it, these are public/system and tmp/pids).

Apparently not many people seem to use an environment like this. It’s pretty old-fashioned in this highly virtualized world, but you run across it from time to time. So what can you do?

Hacking the Symbolic Links

This isn’t going to be pretty. To get the thing to work somehow I created a filter for after_update_code and removed the links created by Capistrano to replace them with new ones, only this time they wouldn’t use absolute paths, but relative ones.

I’m not proud of this solution, but I had to come up with something pretty quickly, and it works for the moment, and was only supposed to do so. It will be replaced with a production-like deployment soon. I’ve replaced the task :symlink with my own which looks like this:

desc "Overwriting symlink task to 
task :symlink do
  on_rollback {
    run "cd #{deploy_to} && ln -nfs releases/#{File.basename previous_release} current"

  run "cd #{deploy_to} && ln -nfs releases/#{File.basename current_release} current"

  run <<-CMD
    cd #{deploy_to} &&
    rm -f current/log current/public/system &&
    ln -nfs ../../shared/log current/log &&
    ln -nfs ../../../shared/system current/public/system

  run <<-CMD
    test -d #{shared_path}/pids && 
    rm -rf #{release_path}/tmp/pids && 
    cd #{deploy_to} &&
    ln -nfs ../../../#{File.basename shared_path}/pids current/tmp/pids; true

One remark: I remove some symlinks in the code, because Capistrano creates them in the update_code task.

As I said, it’s not pretty but it works. Here’s what it does: It removes the symbolic links created by Capistraon and replaces them with new ones using relative paths.

Putting the user in a cage

The better solution is to directly deploy into your chroot environment. Create a user that’s directly being send into it by SSH or by its login shell.

This scenario requires that all the tools needed for a deployment must be installed in the chroot environment, including Subversion. If that’s not a problem, this might be the ideal situation.

One thing you can’t do is restart your web server from here. A scenario like this would mean that you can only restart your FastCGI or Mongrel processes. This is a scenario I’ll use in the long run.

So is this hassle worth it? I’m not sure myself. It’s questionable, if the little added security is worth the effort you have to put in to get your applications working. In the end it depends on what your client's environment looks like, and if you have any control over it. If guidelines require chroot’ed services, then there’s not much you can do. Other than that I’d consider breaking the service out of the chroot and trying to find a better way of securing it. Xen and the like come to mind.

Tags: rails, ruby

In an earlier post I wrote about namespacing your Rails model. There's an additional issue that must be thought of when doing so. Rails (of course) has conventions dealing with namespaces.

Normally, when you put classes in namespaces, like Admin::User, Rails normally expects this class to be in a folder admin in your app/models directory. Responsible for this is the method load_missing_constant in the module Dependencies, part of ActiveSupport. It also uses the namespace to build the table name. So Admin::User would result in admin_users. This isn't always a desirable outcome. Of course you could set a table name explicitly:

class Admin::User < ActiveRecord::Base
  set_table_name "users"

If you need to avoid namespace clashes, that's an acceptable option. But what if you only want to bring some order to your model directory? You want to create some subfolders to separate your model physically, if only to avoid having dozens of files in the model folder.

This is where load_missing_constant kicks in again. If you don't load the files in the subdirectories explicitly, it will assume that the files are in their according namespace. So having a class User in a file user.rb lying in the folder app/models/admin/ will lead it to assume User exists in the module Admin. To avoid that you'll have to add your model subfolders as load paths. To do that you can add the following line to your environment.rb:

config.load_paths += Dir["#{RAILS_ROOT}/app/models/[a-z]*"]

This will tell Rails to look for the the files in all subfolders of app/models on startup. This won't solve the issue yet, you'll still need to explicitly load the classes. So you put the following lines at the end of your environment.rb:

[ "app/models" ].each do |path|
  Dir["#{RAILS_ROOT}/#{path}/**/*.rb"].each do |file|
    load file

That way you'll avoid the implicit call to load_missing_constant. You can add directories to the list, e.g. a subdirectory in lib. You could also explicitly require the classes you need, but who wants to do that, really?

Tags: rails, ruby

I had a nice revelation earlier this week, when I finally tested some code I wrote in the wild, the wild being BackgrounDRb. The code used some pretty common class and module names. I didn't expect many problems when I used something along the lines of Sync or Synchronizer. When running the code in test cases there wasn't any problem.

At first I got the error, that I couldn't include Sync (my module) with the include keyword into a class. I wondered why Ruby wanted to tell me that my module would be a class. I quickly found the answer. Ruby's standard library contains a class named Sync. As long as my code ran in a test case that class didn't come into play. But BackgrounDRb uses apparently uses it, therefore it got loaded before my module. That was the first name clash.

The second involved the class Synchronizer which is a class included in the package Slave used by BackgrounDRb. Slave's class even comes along with the same method I used, so you can imagine the look on my face, when I discovered that the code included there immediately started running when my BackgrounDRb worker ran.

That it took me some hours to find these issues because of a bug in BackgrounDRb is a totally different issue.

The moral of the story: namespace your model classes, and the code in lib/. The more classes you have, the bigger the chances that it clashes with a class in one of your installed gems. The best example is a class like User. The chances are pretty good that it's being used somewhere else. To avoid a clash choose a namespace corresponding to your business case or application (the project name would do).

Keep in mind that Rails expects a directory structure corresponding to your namespaces. So if you have a class Bacon in the module Chunky, be sure to put the class file bacon.rb into a subfolder named chunky in the app/model directory. The same goes for code in lib/.

For smaller projects this might not be necessary, but I learned the hard way that it can be a pretty good practice.

Tags: rails, ruby

While setting up a new project with Rails 1.2.3 I ran across a weird issue in my tests. Somewhere from the CGI class (belonging to Ruby's core lib and extended by Rails) a nil object was used and threw an error. Turns out this was introduced in 1.2.2 with a fix that was supposed to improve CGI handling on Windows.

The issue has been fixed in the stable branch of Rails 1.2, but if you're stuck with 1.2.3 or 1.2.2 you can do what I do: use a plugin. I took this opportunity to write my first, so here it is.

Sorry I don't have a fancy SVN set up for your plugin-script-using pleasure, but this is a tiny fix anyway.

Though this was a rather annoying bug the fix emphasizes the beauty of Rails and Ruby: It's easy to extend and even to fix.

Tags: rails, ruby

Kudos to Dr. Nic for this one. At least a discussion filled with arrogance and finger-pointing turned out something useful.

I worked on a similar problem two weeks ago and came up with a pretty similar solution, though without the fanciness of just defining new databases in database.yml and having the classes in corresponding modules automatically.

To sum up: no jealousy, I'm just glad other's have taken a very similar approach. I just wish, Alex Payne of Twitter would've given this interview a little earlier.

Tags: rails, ruby