Two popular CI services have just implemented Flowdock integrations: Semaphore & TravisCI. Flowdock goes great together with continuous integration apps. That way the whole team can take responsibility of the state of the build. Previously, Flowdock has CI integrations for CircleCI, Bamboo, CruiseControl and Jenkins.
Special thanks to Phil Cohen for implementing the Travis CI integration.
Our lead developer Ville Lautanala gave a conference talk at Frozen Rails about Flowdock’s distributed server architecture and how we use Chef and ZooKeeper to coordinate a set of services.
You can find the slides from SlideShare:
Thanks to all Frozen Rails organizers, once again it was a great conference!
The new asset pipeline is the single most important new feature in Rails 3.1. DHH used most of his keynote at RailsConf 2011 to explain its new features. In this blog post, I’ll go through its implications to an existing application and explain how we integrated it to our workflow at Flowdock.
The Rails 3.1 asset pipeline integrates well with CDNs like Amazon’s CloudFront: just point your distribution to your web server. Most likely, you won’t need your old hacks for assets.
The new asset pipeline in Rails 3.1 makes assets first class citizens in Rails. Assets are moved from public directory under apps/assets and they receive much more attention from Rails. Asset files can inline other asset files using requires and gem dependencies can provide asset files for the application to use.
This means that you don’t have to include jquery.js to your repository. Generators don’t have to worry about creating proper static assets under your public folder. Assets from other sources work just like those which are under your app/assets folder.
The static asset URL scheme has changed. Compiled assets are served under /assets. In production mode, generated files include an MD5 hash as part of the file name, which is a better cache buster than the previous timestamp scheme. Asset pipeline also generates URLs inside stylesheets so that they can link to generated files.
A stylesheet stored at app/assets/application.css will be served as /assets/application.css. Most likely, this layout will contain dependency declarations such as inclusion of CSS reset. Thus, your new application.css would become
At the heart of the pipeline is Sprockets 2.0. In addition to dependency management, Sprockets will allow any number of filters to be used with asset files. By adding extensions to the files names one can control which filters are used. For example, it is possible to run a file through SASS and ERB by naming it stylesheet.css.scss.erb. JS files can also be filtered using both CoffeeScript and ERB.
Among the more controversial changes, CoffeeScript and SASS are now included in the default template. It is possible to use both CoffeeScript and SASS with old style assets. Not much has changed in this respect, DHH just is more vocal about what you should use.
Additionally, support for JS minification comes built-in with Rails. The default is to use UglifyJS via the Uglifier gem. Existing apps can use this by adding it in their Gemfile and setting the minificator in the (production) environment configuration.
Gemfile:
config/enfironments/production.rb:
Running rake rails:update will show how your configuration files should be updated for them to work with new asset pipeline. At a bare minimum, you’ll only need to add
to your application.rb. After that stylesheets, javascripts and images can be moved under app/assets.
There are a few things to change to make your assets work. All absolute paths to images in stylesheets and asset helpers are broken and should be replaced with call to relative helpers, e.g. image_path /images/subdir/rails.png should be asset_path "subdir/rails.png". This way Sprockets will be able to figure out where the asset is served. It is especially neat in production where file names will have MD5 hash suffix for more efficient caching.
You’ll also want to put this to your .gitignore file:
Serving assets in production environment is much more pleasant with the new pipeline. First of all, instead of a timestamp, an MD5 hash is used to bust caches. Second, this is not part of the query string but used as suffix in file name. These MD5 suffixed files really exist and you can have multiple versions coexist at the same time.
Because the URIs are unique not only by query string but also file name, CloudFront can be easily used to serve your assets. Create a new custom origin distribution with your host as origin DNS and use the CloudFront domain as your asset hosts. That’s really all you need barring some web server configuration: check CloudFront custom origin best practices.
One thing to notice is that the assets should be precompiled before your site goes live to reduce load. There is a rake task for this, @rake assets:precompile@. We put this in our deploy.rb:
If you have multiple CSS and JS files, you might have to specify manually which of them are precompiled by adding them to the config.assets.precompile array.
Compiling your assets require that you have a JS runtime available if you use either CoffeeScript or Uglifier. We use therubyracer, but your mileage may vary. Installing node.js is also a good alternative.
Is this really better than the old setup? Did we learn anything?
First, the development and production setups are now more similar. Previously multiple JS files containing our application code were loaded in development, but now they’re concatenated to one big file. We are more likely to run into weird bugs before code is deployed to QA, but on the other hand line numbers aren’t meaningful for debugging purposes. When using CoffeeScript, this point is moot.
Second, we are happy that we could remove our old hacks to get Flowdock deployed. Instead of the combination of patched cloudfront_asset_host gem and some other hacks, we can compile our assets to be served via CloudFront using the built-in rake task.
Third, our deployment times were reduced by over a minute as we don’t need to sync our asset files with the S3 bucket.
Long story short: Ruby on Rails session handling does have some concurrency problems. Especially with AJAX requests this is a very potential scenario:
Now, of course whatever request 1 wrote there is pretty much lost. These bugs are hard to find, and in my case the site had been in production use for 18 months before this really became a problem.
I soon found out that Frederick Cheung had already written a plugin called Smart Session Store to handle these race conditions by locking the session row when needed. However, it didn’t work with the latest PostgreSQL drivers.
I’ve fixed this to work with combinations of the pg/postgers driver and Rails 2.2/2.3 in my smartsessionstore fork. The tests are also now actually testing something. I’ve sent a pull request, so everyone should be able to enjoy it soon.
It’s really a good practice to somehow handle session concurrency even before it’s a problem, because these kinds of bugs tend to appear when you least expect it.
We can test this by adding two actions to our controller. They should both write something to the session, and the other one should sleep a while to make sure that its changes are likely to be lost.
The Firefox plugin, Ubiquity, is an awesome extendable Swiss knife command line of the web. The other day I noticed that Jack Dempsey had written a Ubiquity command for searching the Ruby documentation from APIdock. After quick googling, I found out that different people had done the same for Rails and RSpec as well, based on Jack’s command. Talk about community effort! Here are links to the Ubiquity commands (you need to have Ubiquity installed):
I must not be the only one who occasionally wants to stop a Ruby on Rails application. There are long-running and risky database schema migrations and data migrations, and you don’t want users fiddling the system in the middle of the deployment.
With a Mongrel-based setup it was easy to set up a 503 (Service unavailable) error page and then just shut down all the Mongrels, so that Apache could give users a maintenance page. Using mod_rails (Phusion Passenger), only restart is supported out-of-the-box.
However, it’s possible to use mod_rewrite to prevent users from accessing your site during the deployment. Try this in your Apache virtual host configuration file:
# This option is not needed with Passenger >=2.1.1.
RailsAllowModRewrite on
ErrorDocument 503 /503.html
RewriteEngine on
RewriteCond %{DOCUMENT_ROOT}/../tmp/stop.txt -f
RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ /$1 [R=503,L]
Basically the last three lines mean:
It doesn’t actually stop your Rails processes, though. They will die when Passenger times out.
Now it’s also easy to incorporate this to your Capistrano configuration:
namespace :passenger do
desc "Restart Passenger"
task :restart, :roles => :app do
run "touch #{current_path}/tmp/restart.txt"
end
desc "Stop Passenger"
task :stop, :roles => :app do
run "touch #{current_path}/tmp/stop.txt"
end
desc "Start (or un-stop) Passenger"
task :start, :roles => :app do
run "rm -f #{current_path}/tmp/stop.txt"
end
end
After this it’s safe to deploy big new releases without completely taking down your Apache.
In addition to all kinds of general Ruby fame, Dr. Nic is the creator and maintainer of the Ruby on Rails Textmate bundle. Now, it seems that just a few hours ago he noticed that our Ville Lautanala aka Lautis had written a few good bugfixes and changes to the bundle. He then went ahead and merged them to the master branch. And the real kicker here is, that among those changes was the change that adds APIdock-powered documentation macros. Let’s hope that this change sticks so all Rails codin’ Textmate users will be able to enjoy the rich documentation browsing that APIdock provides.
Will the Ruby bundle, which is also maintained by Dr. Nic, follow next?
Update: Origin of the bundle can be changed using following commands:
% cd ~/Library/Application\ Support/Textmate/Bundles/Ruby\ On\ Rails.tmbundle % git remote rm origin % git remote add -f -t master -m master origin git://github.com/drnic/ruby-on-rails-tmbundle.git
First came Vim, and next Gaizka added APIdock integration to Emacs. Now perhaps the biggest favorite of most Ruby and Rails developers, myself included: Textmate! Courtesy of one of Nodeta’s own, Ville Lautanala aka Lautis, you can get the Textmate bundles with APIdock goodness for Ruby and Rails from github with just a couple of simple git commands:
Now, the keyboard shortcut
gets you the page for the current word from APIdock.
As the Rails documentation discourse is really bubbling, our schedule has given us the sweet chance of taking a few steps back and let us concentrate on the first release of APIdock.
APIdock will be deployed today and we’ll import different versions of the included projects slowly for the next couple of days. The app will be completely usable during the version roll-out. Here are some of the most important changes from Rails-Doc.
The most important difference of APIdock in relation to Rails-doc is of course multiple projects. You will be able to surf your way to APIdock.com and search and browse Ruby and RSpec documentation (in addition to Rails) with the same (except for the improvements that we’ve made) interface that you have been able to use in Rails-Doc. To begin with, the newest patch level of Ruby 1.8.6 will be included as we slowly roll older versions in. Ruby 1.9 will follow later if there proves to be a demand for it. Users wont yet be able to add their own projects, but we’ll provide an easy way to suggest new ones to be added.
In this first release, all the included projects will be listed in tabs, but later on when more projects are added, the idea is that users will be able to choose their “favorite” projects that will be shown as tabs. This way the app will be custom-made for each user.
We’ve added some project specific stuff like extensive project details and version history of the added versions. Behind the curtains the importing of new versions is done with a web interface.
We have also made our ACL more complex under the hood. We can now have moderators that have some extra rights like editing other users’ notes. This way we can give moderator rights to other people including some of our most active collaborators. If you’re interested in becoming a moderator, please contact us at team@apidock.com.
Your Rails-Doc accounts will be preserved in APIdock, the notes will be where you wrote them and the thanks you’ve got won’t disappear either. Any URIs to the rails-doc.org domain will redirect to the correct page under apidock.com. There aren’t any drawbacks to the migration – no functionality is lost. The app was designed to support multiple projects right from the get-go and now that decision is paying off.
The summer is starting to be over and that means the super-active development cycle of APIdock will slow down. We will continue to maintain the app, fix any bugs that are found and concentrate only on absolute key features.
Hopefully you’ll enjoy this first installment of APIdock. In any case, let us know what you think.
Our planned schedule and features held! Rails API documentation with versions support! Full search! OpenId support! Gravatarrrrs! Check it out!
Seriously though, the deployment was delayed by a small surge of bugs found right before the intended deployment window, so we are about 20 hours late. It took about four hours because of all the version-specific documentation generation. We are sorry for any inconveniences you may have experienced and we hope you’ll enjoy this installment of Rails-doc.
