I need more followers on twitter :-)

Today I got my Canadian citizenship.

Both Germany and Canada allow multiple citizenships so I’m allowed to carry both. Geographical redundancy accomplished. Next step is to apply for my Canadian passport so that I can instant failover :-)

Update: Take the citizenship test yourself and post your score in the comments.

Todd Hoff in his love letter to Amazon’s SimpleDB

SimpleDB shifts work out of the database and onto programmers which is why the SimpleDB programming model sucks: it requires a lot more programming to do simple things. I’ll argue however that this is the kind of suckiness programmers like. Programmers like problems they can solve with more programming. We don’t even care how twisted and inelegant the code is because we can make it work. And as long as we can make it work we are happy.

I can’t think of what drove him to write this. This argument is directly contradicted by the success of Rails.

James released his development version of Active Shipping to github

Active Shipping does what Active Merchant did for payment gateways: Provide one unified API to talk to all the Parcel Services on the web. Given a weight, a to and a from address you can calculate the Shipping costs of every supported Shipping Service. All the complexity of the task is handled by the library.

Active Shipping has been in production use with Shopify since the beginning of the year.

require 'active_shipping'
include ActiveMerchant::Shipping

# Package up a poster and a Wii for your nephew.
packages = [
  Package.new(  100,                        # 100 grams
                [93,10],                    # 93 cm long, 10 cm diameter
                :cylinder => true),         # cylinders have different volume calculations

  Package.new(  (7.5 * 16),                 # 7.5 lbs, times 16 oz/lb.
                [15, 10, 4.5],              # 15x10x4.5 inches
                :units => :imperial)        # not grams, not centimetres
]

# You live in Beverly Hills, he lives in Ottawa
origin = Location.new(      :country => 'US',
                            :state => 'CA',
                            :city => 'Beverly Hills',
                            :zip => '90210')

destination = Location.new( :country => 'CA',
                            :province => 'ON',
                            :city => 'Ottawa',
                            :postal_code => 'K1P 1J1')

# Find out how much it'll be.
ups = UPS.new(:login => 'auntjudy', :password => 'secret', :key => 'xml-access-key')
response = ups.find_rates(origin, destination, packages)

ups_rates = response.rates.sort_by(&:price).collect {|rate| [rate.service_name, rate.price]}
# => [["UPS Standard", 3936],
#     ["UPS Worldwide Expedited", 8682],
#     ["UPS Saver", 9348],
#     ["UPS Express", 9702],
#     ["UPS Worldwide Express Plus", 14502]]

# Check out USPS for comparison...
usps = USPS.new(:login => 'developer-key')
response = usps.find_rates(origin, destination, packages)

usps_rates = response.rates.sort_by(&:price).collect {|rate| [rate.service_name, rate.price]}
# => [["USPS Priority Mail International", 4110],
#     ["USPS Express Mail International (EMS)", 5750],
#     ["USPS Global Express Guaranteed Non-Document Non-Rectangular", 9400],
#     ["USPS GXG Envelopes", 9400],
#     ["USPS Global Express Guaranteed Non-Document Rectangular", 9400],
#     ["USPS Global Express Guaranteed", 9400]]

Git support in the latest capistrano works very well but there are two gotcha’s I ran into, i’ll document them here so that Google can pick it up.

The first problem was the bizarre error message I got when I forgot to push my changes to the deployment repository:

 ** [err] Needed a single revision
 ** [err] Needed a single revision
 [...]

Needed a single revision, well then. Git seems to use the rare other meaning of “single” which means “existing” or “valid”.

The other problem was harder to track down. Capistrano would simply hang forever after the update_code task. It would also leave the cached_copy directories in a totally invalid state on some servers which required manual rm -rf. Thanks go to Scott Raymond on Twitter for pointing me in the right direction. It seems that we have too many App servers for a stock sshd_config and It triggers some kind of throttling logic which git cannot deal with.

If you are deploying to medium to large server farms better head over to your git box and increase the MaxStartups now.

MaxStartups 10000:1:10000

This solves the problem.

Are you in Ottawa, Canada or willing to relocate? Want to work in the coolest office in town? Would you like to work on a high profile rails application with millions of users? Do you want to work with some of the best rails programmers out there?

Please email me at tobi@jadedpixel.com. Please include a piece of code you are especially proud of or point me to some of your open source work. If you include a CV please attach it as PDF.

if false
  var = nil
end

p var.nil? #=> true ??? 

ActiveSupport has this really nice returning tool. It allows you to rewrite the common pattern of initialization and returning a object from am method like this:

returning something = Thing.new do
  something.this
  something.that
end

We were wondering if there may be a performance issue with the tool so a quick benchmark showed the following:

                                          user     system      total        real
def n; i = 0; return i; end           0.510000   0.000000   0.510000 (  0.520417)
def n; i = 0; i ; end                 0.340000   0.000000   0.340000 (  0.344569)
def n; returning i = 0 do; end; end   0.730000   0.000000   0.730000 (  0.726853)

Those numbers are after 1m runs on a core 2 duo laptop. As you can see the difference is very minor. Code vanity can safely prevail. Full Benchmark

The quality of Shopify stores being launched every day really boggles my mind. Just look at those 3 most recent examples:

ClothMoth

ZipRC

Destinative

If you like pretty stores, i recommend subscribing to the Shop of the moment feature or head over to our http://www.shopify.info/screenshots/ page.

Speaking of which, our screenshot page is actually powered from the RSS feed of the Shop of the moment category on our blog. Shopify.info is powered by radiant and we use the RSS Reader extension with some minor modifications and some clever CSS. This is a great example of how we can reduce content production work by re-using various RSS feeds. Another example of this is our feature page where we list all the Payment gateways Shopify currently supports. This comes straight from a RSS feed generated by the Shopify application and so it’s automatically always up-to-date.

Paul Grignon’s 47-minute animated presentation of “Money as Debt” tells in very simple and effective graphic terms what money is and how it is being created.

via google video :

I just checked in a new version of the delayed_job plugin which handles background processing of long running tasks in Shopify.

DJ is now fully parallelizable and does not require any global locking anymore. This means that you can run as many worker processes as you want across your server farm if you need to speed up the queue processing.

This became necessary when we kicked off a full search server reindex recently and realized that a single worker process would require 48 hours to complete the task. Such is the burden of success.

This feature is DB independent and doesn’t rely on row level locking. I found that row level locking lead to a lot of unnecessary lock timeout waits. If you are updating from the previous version of the plugin please be advised that there are two new columns you have to add.

Grab the latest version form http://github.com/tobi/delayed_job/tree/master

Small plugin for using the global lock service of mysql in your rails app: Locking plugin

Great if you have long running cron tasks or require exclusive access to some resources.

Invoice.acquire_lock("Shopify billing") do  
  # Can only run once across your server farm
  Invoice.find_all_due.each { |invoice| invoice.collect_payment! } 
end

So we went from Darcs -> SVN -> Mercurial -> Git in just over 3 years and without losing a single commit in the process. 4th one is a real winner though. If you don’t use git yet you should really check it out.

If you are unfamiliar with GIT, I recommend watching Peepcode’s introduction. We have a site-license for peepcode at jadedPixel and often run the latest screencasts on the big LCD in the office during lunch break. Its great.

I finally got a github invitation and used the opportunity to release another Shopify extractions.

Delayed::Job or DJ is a asynchronous priority queue which only relies on a simple database table. It doesn’t require you to run a dedicated server like many other systems do.

We use for a lot of longer running tasks in Shopify such as sending newsletters, uploading files to s3, downloading images from urls, indexing products to Solr and so on.

There are two ways to add jobs to the queue:

Jobs are simple ruby objects with a method called perform. Any object which responds to perform can be stuffed into the jobs table. Job objects are serialized to yaml so that they can later be resurrected by the job runner.

  class NewsletterJob < Struct.new(:text, :emails)
    def perform
      emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) }
    end    
  end  

  Delayed::Job.enqueue NewsletterJob.new('lorem ipsum...', Customers.find(:all).collect(&:email))

There is also a second way to get jobs in the queue: send_later.

  BatchImporter.new(Shop.find(1)).send_later(:import_massive_csv, massive_csv)                                                    

This will simply create a Delayed::PerformableMethod job in the jobs table which serializes all the parameters you pass to it. There are some special smarts for active record objects which are stored as their text representation and loaded from the database fresh when the job is actually run later.

The plugin can be found on github.

If you are working on a ruby application that requires dealing with credit cards, you are probably using ActiveMerchant. If not, you probably didn’t know about ActiveMerchant.

ActiveMerchant is an extraction from Shopify. It’s a simple to use library which translates one common interface into the wire language of 30-40 different payment processors around the globe with more added at rapid pace. As long as your application can talk to active merchant you can switch payment providers with a single line of code.

Treat yourself to Cody Fauser’s excellent ActiveMerchant PeepCode PDF which is an in depth discussion about the library and covers topics such as order pipelines, order state management and the appropiate unit testing which a financial application requires.

Cody is the main programmer ActiveMerchant which I originally started. Cody took the library further than anything I envisioned and it’s now one of the most competent libraries for ruby.