Tag: ruby

 

Experiments in metaprogramming

I’m having a go at metaprogramming in ruby. It started out simply enough. I wanted a module which had some custom assert methods in. Here’s the test:

  def test_matches
    assert_stderr_matches(/foo/) do
      warn "foo"
    end
  end

And here are the custom assert methods.

  module AssertStderr
    def assert_stderr_matches(regex, message=nil)
      old_stderr = $stderr
      $stderr = StringIO.new
      yield
      assert_match(regex, $stderr.string, message)
    ensure
      $stderr = old_stderr
    end

    def assert_stderr_empty(message=nil)
      # More-or-less identical.
    end
  end

Two methods, the bulk of which is similar. Yuck. Refactoring time.

  module AssertStderr
    def assert_stderr_matches(regex, message=nil, &block)
      assert_match(regex, capturing_stderr(block), message)
    end

    def assert_stderr_empty(message=nil, &block)
      assert_equal("", capturing_stderr(block))
    end

    def capturing_stderr(block)
      old_stderr = $stderr
      $stderr = StringIO.new
      block.call
      return $stderr.string
    ensure
      $stderr = old_stderr
    end
  end

That’s better, but it set me wondering how far I could go down this route. What about creating an assert_stderr builder? This is what I’ve come up with so far.

  def self.assert_builder(what, &check_block)
    define_method("assert_stderr_#{what}") do |*args|
      begin
        old_stderr = $stderr
        $stderr = StringIO.new
        yield
        check_block.call($stderr.string, *args)
      ensure
        $stderr = old_stderr
      end
    end
  end

  assert_builder :matches do |stderr,regex,message|
    assert_match(regex, stderr, message)
  end

  assert_builder :empty do |stderr,message|
    assert_equal("", stderr, message)
  end

It looks reasonable, but it doesn’t work. It’s meant to create a method which takes a block and executes it. However, careful debugging (read: p) shows that the yield is actually calling the check_block parameter instead of the block that’s provided when the assert_stderr_matches method is called. At this point, I have no idea why this is happening. I’m very confused.

It doesn’t help that you can’t really google for “ruby blocks that take blocks” and get a sensible answer.

Maybe I just need to nose around the Rails source long enough. There are examples of most forms of Ruby magic hanging around in there…

Coding Dojo 9

I’m just back from Dojo#9. We attempted Kata 4 again, but in Ruby this time. Even though I was the only one with much Ruby experience behind me, it still wasn’t much of a problem in practise. I did some cheat sheets, but they didn’t seem to be used much.

Exactly the same as the last time, we only completed the first part of the kata. I started the session by writing a small amount of code. In a very non-object oriented fashion. However, I honestly believe that for this problem an OO solution was utter overkill. The testing was probably useful, but to be quite frank a program this short can be visually inspected so easily, I hardly feel it’s worth it. Heresy, I know.

This is what I was aiming towards.

  day = nil
  min_spread = nil
  File.foreach(ARGV[0]) do |line|
    if line =~ /^\s*(\d+)\s+(\d+)\s+(\d+)/
      spread = $2.to_i - $3.to_i
      if !min_spread || spread < min_spread
        min_spread = spread
        day = $1
      end
    end
  end
  puts day

I’m pleased about using regexes though. They make the problem simpler (and more robust) than the previous Java solution using substring().

Now, when it came to refactoring the program to get to the third part of the kata, an OO solution would have been more appropriate. Because you need to carry some state around, and you can obviously use inheritance to model the differences. And by that point, you’ve got two working programs you can extract a sensible design from.

Ah well. An entertaining evening. Big thanks to Joh for organising and FP for hosting. Sorry for disturbing the late workers…

Dojo VIII

Last night was the 8th Coding Dojo. Joh kindly allowed myself and Jay to try out one based on Ruby instead of the usual Java. Sadly, Jay was ill and couldn’t make it. Get well soon, Jay.

We started late because I was installing all the necessary bits (ruby + eclipse) on Joh’s laptop…

Having done that, I gave a quick presentation comparing Ruby to Java. I had been under the impression that most people there would be familiar with Java (based on previous experiences). Unfortunately, because it was Ruby it attracted a very different crowd. Thankfully, nobody seemed to mind too much.

After that, we started on the task: write a Sudoku solver. This was the point where I realised that whilst I’d brought along code for a board, I hadn’t actually brought a puzzle to solve. Ooops. So we hopped over to Wikipedia to grab a sample puzzle. And we then realized I needed an importer… So we knocked one up fairly quickly. Whilst I did feel quite embarrassed about lacking such a crucial part of the problem (bad prep! no cookie!), it turned out to be an interesting example of coding something simple. The fact that I was with Andy, who I know well helped a lot.

Unfortunately, as the evening progressed, it became rapidly clear that none of us had much clue about how to solve a Sudoku puzzle algorithmically. So most of the time ended up being spent discussing the problem as opposed to coding. Afterwards, it was suggested that a “time out” be taken so we could discuss the strategy as a group—that might have helped quite a bit.

Finally, the last pair on the keyboard started making real headway. Andy and James came up with an algorithm for listing the “opportunities” for a given cell and were in the middle of implementing it when the final whistle blew.

So we didn’t get very far at all towards a solver. But as usual the journey was interesting and entertaining.

class methods in ruby modules

Modules in ruby are fairly simple to understand. You can add extra methods to a class by including a module.

  module Extras
    def say_hi
      puts "hello world"
    end
  end
  class Foo
    include Extras
  end
  Foo.new.say_hi

But one thing that crops up reasonably often (particularly in rails) is the need add not just methods, but class methods.

Now, the ever lovely textmate has a builtin snippet for doing just that. It appears to be a common ruby idiom.

  module Extras
    module ClassMethods
      def say_hi
        puts "hello world"
      end
    end

    extend ClassMethods

    def self.included(receiver)
      receiver.extend(ClassMethods)
    end
  end

The only thing I’ve added is the say_hi method. When included in class Foo, it lets us say Foo.say_hi

But what’s going on here? There are quite a few moving parts:

  • We define a second, nested module to hold the ClassMethods.
  • We extend Extras with ClassMethods. This is really Object.extend. So what’s the current object that this is being called on? Well, it turns out to be an instance of Module. This means that we’re pulling in all the methods in Extras::ClassMethods into Extras as well. But they’re defined as class methods, so you can say Extras.say_hi.
  • The next method, Module.included is a callback. It gets called whenever a this module is included by another class. It gets passed in a reference to that other class.
  • Inside included, we then extend the class that included us with Extras::ClassMethods. This makes all the class methods available in the class that included us.

This is all possible thanks to Ruby’s extremely open and consistent object model.

Brighton Ruby Users Group

Last night was the inaugural meeting of the Brighton Ruby User Group at the Eagle. I gave a (slightly) slimmer version of my Unicode for Rails talk1. I really should learn about code on slides though. Just because something looks good on screen, the lovely soft focus effect that projectors provide quickly turns it to mush…

Jay Gooby also gave an excellent off-the-cuff introduction to some of the neat bits which are up & coming in Rails 1.2 (simply_restful in particular). rake rails:freeze:edge is the magic. And if you get bored, just rm -rf vendor/rails.

But it’s the people that make the evening. And we had a superb turnout—at least 14 people. There were lots of faces I hadn’t seen before, which is excellent. Also, a very wide range of Ruby / Rails experience from “none” to “lots”. In retrospect, I hope I didn’t scare anybody with the deep technical stuff in my talk… Next time I think an “intro to Ruby” talk might be a good idea, judging by the upcoming.org comments.

Anyway, my thanks to James for organising. I’m looking forward to the next one!

1 Slides are over here.

Brighton Ruby Users Group

James McCarthy is taking a stand! He’s started up Brighton Ruby. The first meet is 7pm, Jan 9th at the Eagle. Ideas are being solicited for talks. Do drop a comment onto the blog if you’re going to attend or even better are willing to talk!

I’m really glad to see this happening. There seem to be quite a few people doing Ruby (largely driven by rails) in the local area, and it’s great to get like minds together and discuss ideas. I wish I was one of them, but instead I’m getting driven deeper and deeper into Java land by work. Which is interesting, but has quite the masochist feel (e.g. learning cocoon). Still, I hope I’ll be able to contribute.

gdb ruby $pid

gdb is my “tool of last resort”. When all other online diagnostics have failed, I know enough gdb to pull out a C level stack trace.

  % gdb $SHELL $$
  GNU gdb 6.3.50-20050815 (Apple version gdb-477) (Sun Apr 30 20:06:22 GMT 2006)
  /Users/dom/320: No such file or directory.
  Attaching to program: `/bin/zsh', process 320.
  Reading symbols for shared libraries ........ done
  0x90006108 in syscall ()
  (gdb) where
  #0  0x90006108 in syscall ()
  #1  0x00054558 in signal_suspend ()
  #2  0x0002c9a8 in waitforpid ()
  #3  0x0002cb4c in waitjobs ()
  #4  0x00012584 in execlist ()
  #5  0x00011c24 in execlist ()
  #6  0x00011874 in execode ()
  #7  0x00026890 in loop ()
  #8  0x0002945c in zsh_main ()
  #9  0x00001d14 in start ()

It’s not terribly informative, but in the past, it’s given me just enough of a clue to start looking at the SSL libraries (for example).

Jamis Buck has gone one better—he’s pulled a ruby stacktrace from a running process. Which seems quite magical to me indeed. I also think that you can turn most of what he’s done into a gdb macro. I’ll have to have a look at some examples

In the past, I’ve resorted installing a signal handler “just in case” to pull out this sort of information. All of my Perl apps have this in their startup files.

  $SIG{ USR2 } = sub { Carp::cluck("Caught SIGUSR2 in $$") };

Which is all very well, if you know what you need in advance. Which is not usually the case.

Jamis++

Update: Ok, turning it into a gdb macro is dead easy. Save this lot into ~/ruby.gdb

  # A quick hack to show the environment for a Ruby process.

  define printenv
    set $index = 0
    while environ[$index]
      x/1s environ[$index]
      set $index = $index + 1
    end
  end

  document printenv
    Display the environment for the current process.
  end

  define rb_where
    set $ary = (int)backtrace(-1)
    set $count = *($ary+8)
    set $index = 0
    while $index < $count
      x/1s *((int)rb_ary_entry($ary, $index)+12)
      set $index = $index + 1
    end
  end

  document rb_where
    Show the ruby stacktrace.
  end

To use it do source ~/ruby.gdb from the gdb session, and then you get two new commands: printenv and rb_where.

Oh yes, I do know how super-trivial all this would be if only I had DTrace. Roll on Leopard.

Flickr Favorites

I have been steadily accumulating favorites in flickr for some time now, thanks to the lovely feeds that they offer. But I want them as backdrops for my desktop. I didn’t spot anything else with my 30s of googling, so I wrote a small script to dump my favorites into a directory.

I was pleasantly surprised by how easy the flickr.rb library is to get along with. It’s really trivial to use. I suspect that this is a by product of the flickr api design.

My main disappointment was realising how few people make the full size photos available. In the end, I limited the downloader to photos that are at least 600 pixels in width, otherwise I just ended up with very squished blobs on my backdrop.

The results are worth it. Each time I look at my desktop I now have a fresh picture in it. It makes me smile a lot anyway, and that can’t be a bad thing.

del.icio.us api change

I’ve just been caught out by the del.icio.us api change (they’ve just switched off the old api). This broke the backup script I use. It’s a nice script, it downloads each days bookmarks into a sqlite database, ready for reuse.

Anyway, this is the patch I wrote to make it use the new api (over SSL) instead. BTW, top marks to the del.icio.us people for using basic auth instead of coming up with their own scheme. Basic auth + SSL is a really nice, simple way of doing things.

  --- /home/dom/delicious-backup.rb.orig    Fri Aug 11 09:00:39 2006
  +++ /home/dom/bin/delicious-backup.rb    Fri Aug 11 09:09:19 2006
  @@ -26,11 +32,29 @@
                'values (?, ?, ?, ?, ?);'
   insert_tag = 'insert into tags (hash, tag) values (?, ?);'

  -xml = Net::HTTP.start('del.icio.us') { |http|
  -    req = Net::HTTP::Get.new('/api/posts/all', {'User-Agent' => agent})
  +# https://api.del.icio.us/v1/posts/get
  +require 'net/https'
  +http = Net::HTTP.new('api.del.icio.us', 443)
  +http.use_ssl = true
  +http.verify_mode = OpenSSL::SSL::VERIFY_NONE
  +xml = http.start { |conn|
  +    req = Net::HTTP::Get.new('/v1/posts/all', {'User-Agent' => agent})
       req.basic_auth(user, pass)
  -    http.request(req).body
  +    resp = conn.request(req)
  +    raise resp.inspect unless resp.kind_of? Net::HTTPSuccess
  +    resp.body
   }
  +
  +Dir.chdir(ARGV[1] || ENV['HOME'] + '/libdata/del.icio.us')
  +
  +# Clean anything over 30 days.
  +thirty_days_ago = Time.now - (30 *(24*60*60))
  +Dir["*.db"].each do |name|
  +    f = File.new(name)
  +    if f.mtime < thirty_days_ago
  +        File.unlink(name)
  +    end
  +end

   db_name = ARGV[0] || Time.now.strftime("%Y-%m-%d.db")
   SQLite3::Database.open(db_name).transaction { |db|

That’s also got my backup expiry bits in, which is handy, but could probably be done better. BTW, that ruby idiom of indexing the Dir class to run a glob is freakish.

ruby postgres gem issues

I’m trying to install PostgreSQL support into Rails. This should be fairly simple. Unfortunately, there are a couple of “oopsies” along the way.

First up, there are several gems to pick along the way:

  • postgres
  • postgres-pr
  • ruby-postgres

If you’ve a compiler installed, you want ruby-postgres. It’s a fork of the postgres gem, which knows how to find the postgres libraries and headers through pg_config. The postgres-pr gem is a pure-ruby version.

Being foolish, I tried to install the postgres gem:

  % gem install ruby-postgres-0.7.1.2006.04.06.gem

Sadly, this is what happens:

  Building native extensions.  This could take a while...
  ruby extconf.rb install -l ruby-postgres-0.7.1.2006.04.06.gem
  ...
  make install
  make: Nothing to be done for `install'.

Weird1. So I tried putting make into debug mode (set MAKEFLAGS=-d in the environment).

    Finished prerequisites of target file `/opt/domtest/lib/ruby/gems/1.8/gems/ruby-postgres-0.7.1.2006.04.06/./postgres.so'.
    Prerequisite `postgres.so' is older than target `/opt/domtest/lib/ruby/gems/1.8/gems/ruby-postgres-0.7.1.2006.04.06/./postgres.so'.
   No need to remake target `/opt/domtest/lib/ruby/gems/1.8/gems/ruby-postgres-0.7.1.2006.04.06/./postgres.so'.
  Finished prerequisites of target file `install-so'.

Basically, the freshly-compiled postgres.so is older than the installed version. Which isn’t installed, BTW. I checked. How can this be?

After much messing around, I discovered that the reason is that ruby gems compiles in place. So when it comes to install time, it ends up comparing the compiled file to itself. Naturally, nothing happens.

But then, ruby gems runs make clean, causing it to be removed. Grrr! How on earth has this ever worked for anybody?

Anyway, at this point, I’m going to give up and use the postgres-pr gem instead.

[1] I’m not the first to see this—see ruby postgres problems.