Typo Times Fix

After a bit of searching, I’ve found the problems with the odd times on this weblog. It’s a problem with ActiveRecord (part of Rails) and in particular, PostgreSQLAdaptor.

Basically, it computes a default value once, inside Ruby, and then caches it. Which is a bit odd, to say the least.

There are a number of open tickets related to this: 2873, 2877, 2257 and probably more. But looking at them, it seems that the problem will be fixed in Rails 1.1, which should be released “soon”.

Meanwhile, I’ll patch this particular instance of Typo to update the time on create. This is what I’ve used, and it seems to be working ok.

Index: app/models/article.rb
--- app/models/article.rb       (revision 727)
+++ app/models/article.rb       (working copy)
@@ -94,6 +94,9 @@
     self.extended_html = HtmlEngine.transform(extended, self.text_filter)

+  # Work around PostgreSQL adaptor (
+  before_create { |art| art.created_at = }
   def self.time_delta(year, month = nil, day = nil)
     from = Time.mktime(year, month || 1, day || 1)

Index: app/models/comment.rb
--- app/models/comment.rb       (revision 727)
+++ app/models/comment.rb       (working copy)
@@ -29,4 +29,7 @@
     self.body_html = HtmlEngine.transform(body, config["comment_text_filter"], [:filter_html])

+  # Work around PostgreSQL adaptor (
+  before_create { |art| art.created_at = }

Rails Migrations

Recently, I’ve been playing around with Rails and one of the things that has really impressed me is the support for migrations. If you haven’t come across them before, it’s a well defined way of doing agile development with your databases. Your database contains a table schema_info with a column version. And then you define an ordered series of scripts which each contain up and down methods to migrate between versions.

If that sounds complicated, check out the example from the documentation:

  class AddSsl < ActiveRecord::Migration
    def self.up
      add_column :accounts, :ssl_enabled, :boolean, :default => 1

    def self.down
      remove_column :accounts, :ssl_enabled

You’ll notice that we’re not doing any SQL here. That means that as a side effect of falling into line with Rails, you get a level of database portability for free. Marvellous!

All this was prompted by looking at the recipe on migrations in the new Rails Recipes book.

This is not to say that migrations are a panacea, however. As always, by trading SQL for a domain language, you lose access to some features. Foreign key relationships aren’t easy to express in this manner, for instance. But that doesn’t mean they can’t be used—for an example see robby’s post on Rails Migrations and PostgreSQL Constraints. It just ties you to a particular platform. Which is probably no big deal for most people, as they’re likely only using one database anyway.

So far, I’ve had two major tripups with migrations.

  1. The SQLite adaptor doesn’t appear to support some things properly. I think it managed to nuke the contents of a column I was playing with. But I need to look further at this.
  2. Purely my own fault. I had set up a load of migrations in my development environment and I forgot to transfer them to the test environment. I got very confused for a while until I remembered to do that… Again, it’s my own fault because I was running the individual test files by hand, instead of using rake.

But still, compared to everything else I’ve seen, migrations in rails are superb. Particularly after going through similar pains at work in the last week! Rails once again hits the sweet spot, providing just enough framework to get things done easily, without getting in the way too much.

P.S. If you are playing with migrations, I recommend installing Jamis Buck’s verbose_migrations plugin. It gives you a little more feedback on what’s going on. Normally, I like the Unix way of “no feedback until it goes wrong”, but whilst I’m still learning to trust migrations, this plugin is helpful.


Beta Books

I’ve recently purchased a beta copy of Rails Recipes. It’s obviously not 100% finished, but appears to be largely complete. It’s already been really, really helpful to me, in particular the chapter on migrations.

But what’s really, really cool is the link at the bottom right of each page in the PDF. It just says “Report erratum”. Clicking it launches a web browser pointing at the erratum page for that book. It’s the easiest way of giving feedback that I have ever seen.

It’s not just erratum of course. Because it’s still a Beta Book, suggestions are also solicited. So I added a little note about how linking to the docs for migrations would be a good idea. Yes, it’s a very small point. But together, lots of people with lots of small points adds up to a much, much better book. I can’t wait to get my print copy of the final version!


Locomotive / Typo bug

I’ve been trying to get typo working on my mac, using the very shiny Locomotive. Sadly, I kept getting display errors in my sidebar. After spending 3 hours looking at it last night, I’m now pretty sure that it’s a bug in YAML::load, which is called from inside ActiveRecord::Base#object_from_yaml. It works fine on my FreeBSD server running ruby 1.8.4. But under Locomotive (ruby 1.8.2), Instead of getting back a hash, I get some internal YAML class. Hmmm.

Thankfully, a simple patch is all that’s required to get things working properly.

Index: app/models/sidebar.rb
--- app/models/sidebar.rb       (revision 724)
+++ app/models/sidebar.rb       (working copy)
@@ -27,7 +27,10 @@

   def active_config
-    self[:active_config]||{}
+    ac = self[:active_config]||{}
+    # XXX Work around a bug in some versions of Ruby / YAML.
+    ac = ac.value if ac.respond_to? "value"
+    ac

   def html_id

I’m so glad that I imported typo into subversion on a vendor branch. It makes this sort of thing a heck of a lot easier.

I should go and file a bug in the typo trac. But I have to admit to not being sure that it’s up and running properly yet…


Reboots for Users

Jamis Buck has a good article on how to run TextDrive and Lighttpd together. It involves switching the Rails FastCGI processes to be managed independently from the web server. Essentially, the Rails FastCGI runners become independent daemons.

But what’s really useful is the tip in the first comment. It mentions an obscure crontab feature: @reboot. This will let you, as a user, execute a script when the machine boots up. This is dead handy, as it means that your application can be automatically restarted on server boot without having to bother the sysadmins.

More info:


PUTting an URL

Today, somebody at work wanted to PUT some data to an URL. It’s part of an internal web service we use. Should be easy write. In Perl, it’s pretty trivial.

my $ua = LWP::UserAgent->new();
my $req = PUT '', Content => $ARGV[0];
my $resp = $ua->request( $req );
die $resp->status_line, "n" unless $resp->is_success;

That takes the first command line argument and PUTs it Pretty simple stuff. But compare this to the Java way, which is what my colleague was attempting.

public class HttpPutTest {
    public static void main(String[] args) throws IOException {
        URL dest = new URL("");
        HttpURLConnection conn = (HttpURLConnection) dest.openConnection();
        if (conn.getResponseCode() < 200 || conn.getResponseCode() > 299)
            throw new IOException(conn.getResponseMessage());

Firstly, as with all Java, the sheer verbosity is staggering. But there are two things of interest to me here.

  1. The request and response are conflated into one object, an UrlConnection (or variation thereof). There’s no way to retrieve header values on the request once they’ve been set.
  2. Setting the content is done through the most contorted route. Firstly, we have to tell the connection that it’s to expect output (why? I’m doing output things below, it should work it out). Secondly, we have to fetch an OutputStream, marshal our input into bytes and then write those bytes to the stream.

It’s that OutputStream that gets me. Oh, I can see why you’d need it occasionally. But 90% of the time when you’re doing a PUT or a POST, the amount of data is tiny, and you have it all up front. So you don’t need to stream it to the server. You’re paying the price for it 100% of the time, even though it’s only needed 1% of the time, if that. Really, how hard would a setContent() method be?


Character Encodings

There have been a few links around to an article about AJAX and Multibyte Character Support today. He highlights how he fixed a problem by ensuring that he sent the correct HTTP headers for the character encoding he was using. After stabbing being thoroughly mislead by Internet Explorer.

Now the correct character-encoding ended up being UTF-8. But of course, you actually have to turn your data into UTF-8 first. But it’s still the right answer.

Unless you have really, really peculiar clients, you should always be sending out UTF-8 data. It’s a no-brainer.

But it highlights a deeper issue. You simply have to know what encoding your characters are in. It’s no good saying “ASCII” (I can’t type in “£” then?) or Latin-1 (because invariably it’s cp-1252 instead). UTF-8 being a superset of all the others (with the possible exception of some far-east characters that didn’t make it into the HAN Unification process).

If you don’t know what character encoding you have, then you don’t know how to interpret those bytes. Plain and simple. You need to convert your data to the right character encoding on the way in and send it out in right encoding. Because UTF-8 is a superset, you can transliterate if needed, but it’s an expensive solution.

But it seems that many, many web developers haven’t grasped the concept of a character set yet. It’s just another area when “things sometimes go wrong if you type in funny characters sometimes”. But it needn’t be rocket science.

Oh, and the comments about using escape() in JavaScript are way off the mark. That’s just punting the problem, guys.

Finally, it’s worth noting something else related that popped into my radar today. Sending out character sets with a media type of text/* is absolutely necessary, since it’s perfectly legitimate for a bit of proxy software to then translate to a different character set if it wants to. This is why you should never send out xml as text/*, because then the encoding in the xml declaration would be wrong…



I’ve just switched the theme on this blog to Origami, winner of the typo theme contest. Whilst I think it looks good, it feels a bit slow. And I really dislike the Flash technique for replacing the texts of the headlines. It means you can’t select the text properly any more. So the theme may change again shortly. Given my level of artistic inability, it’s won’t be an original creation though.

Meanwhile, Jeremy Keith has given his blog engine a makeover—Everything old is new again. It looks exactly the same as before, but now comes with added searchy goodness. Of course, he used MySQL instead of a real database, but we can forgive small foibles. ;-)

My only minor critique is that a permanent redirect should be set up on the RSS feed to direct clients towards the new URL. Something like this in .htaccess should do the trick.

  Redirect permanent /journal/journal.rss

I never understood why Apache required you to specify an absolute URL for the destination there. It should be able to work it out if you’ve left the hostname and scheme off…


A Thing of Beauty

There’s an interesting article, A thing of beauty in the latest New Scientist. It has deep resonance with me as a programmer. It reminds me of a quote I once read.

Code should be written for humans to read, and only incidentally for computers to execute. Introduction to SICP

Sorry, I forget the attribution. Knuth maybe? But the point is that elegant code clearly expresses the ideas that you are thinking to other people who read that code. Often, the elegant solution to a problem will also be the naturally faster one as well. Unfortunately, that reminds me of another quote.

For every complex problem, there is a solution that is simple, neat,
and wrong. —H. L. Mencken

But that certainly shouldn’t stop you from trying to find a clean elegant solution. 🙂


XML::Genx 0.20

I pushed out a new release of XML::Genx last night. Nothing worth upgrading for, it just fixes a few compiler warnings under gcc 4.0. Thankfully, after some sage advice from Daniel Jalkut, I managed to get rid of the warnings I was complaining about. Unfortunately, there’s still one warning coming out. But it’s in code that is entirely generated by xsubpp, so there’s not a lot that I can do about it. Ah well.