Go interfaces are for consumption

After a recent discussion, I’ve realised something about Go’s interfaces. They’re best if you use them as a consumer rather than producer. Using my own code as an example. I have a storage layer that uses bigtable.

package storage

type IO interface {
  Read() (string, error)
  Write(string) error

type bigtable struct {…}

fun New() Bigtable {
	return &bigtable{…}

func (bt *bigtable)Read() (string, error) {…}
func (bt *bigtable)Write() error {…}

So, we’re creating an interface then returning an instance of it. This allows us to create a fake version simply.

package fakestorage

type Fake struct {
	value string
	err   error

func (f *Fake)Read() (string, error) {
	return f.value, f.err
func (f *Fake)Write() error {
	return f.err

The consumer just takes the interface.

package logger

type Service struct {…}

func New(io storage.IO) Service {
	return &Service{io}

So what’s the problem here? Well, we’re constraining the output of storage.New() with an eye towards the Service consumer. But there’s no need. The Bigtable struct would adequately satisfy the interface. By returning the interface we can only call the specified methods. There’s no possibility to call something that (e.g.) we only want in startup.

As a bonus, if we return the concrete type, we get far better results in godoc.

Logging in Go

Dave Cheney has an excellent post, Let’s talk about logging. In it, he dissects the current logging libraries, and what you really need. I pretty much agree with everything he says.

  • Warnings are just more info messages.
  • Log or handle the error — not both.

Where I disagree is with the error logging. It is distinct from info logging. It just needs to be actionable. if there’s an error, somebody needs to do something to fix it. See also Rob Ewaschuk’s Philosophy on Alerting.

As an aside, I’m glad to see that we’ve open-sourced our internal logging library as glog. It certainly doesn’t meet the criteria above, but it’s nicer than the standard library in a few small ways (like Verbose for developer-debugging logs).

Go Readability

If you haven’t seen them, do take a look at these slides on Go readability, from one of Google’s Go readability group.

Readability is an important process at Google. In theory, it’s about ensuring the style guide for a language is applied. In practice, it’s also about ensuring that idiomatic code is produced. This is highly language specific, and not something that can easily be done with tooling.

In the case of Go readability, it feels like a mentoring process over a series of code reviews (other languages take a more “big-bang” approach). The end result is that I have a better idea of not just how to write Go, but how we like Go to be written at Google. I really appreciate the strong emphasis on simplicity in Go code. Hopefully, that comes through in the slides.

Rescuing Data with F-Script

I have an old iMac. It’s not in a good state.

Unwell iMac

I don’t care that much, it’s old slow, and not a lot of use. The only remaining thing of value is the large collection of photos I have on there. I have an external drive, so it’s easy to copy them over.

All the photos are stored in iPhoto 5, which (from googling around) appears to be the version before they moved all the metadata into a big binary blob. And looking in the iPhoto Library folder revealed a file AlbumData.xml which contained details of all 12,000+ photos. Result!

Apart from one slight problem. I’d never used iPhoto’s “albums.” Well, I’d made one or two (mostly by accident). But for the most part, I’d left the photos in the import “rolls”, just renaming (e.g.) “Roll 42” to “Trip to Wales”. Now the rolls are represented in AlbumData.xml, but without the roll names. Just “Roll 1”, “Roll 2”, etc. <facepalm/>

So like any good engineer, I’ll spend a huge amount of time to avoid having to re-enter 340 roll names, which I could have done in a couple of hours. This information is necessary for whatever system I’ll be pulling the photos into afterwards, dammit!

iPhoto obviously knows what the name of each roll is, or it wouldn’t be able to display them. They’re just stored in some different place (a binary blob, as it happens). Thankfully, I can still run iPhoto. So long as I avoid the middle of the screen, anyway. Now I forget exactly where I heard about it (Core Intuition, perhaps?) but I remembered that F-Script Anywhere allowed inspecting a running Cocoa program.

F-Script itself is a smalltalk based language, built on top of Objective-C and the Cocoa runtime. The tutorial includes a nice side-by-side comparison with Objective-C. It’s pretty simple for the most part. F-Script Anywhere is an add-on that allows you to inject the F-Script runtime into any running process. I tried it with iPhoto and immediately had a command line and browser with complete access to all of iPhoto. It helped having a minimal knowledge of Cocoa (I could find the app delegate!)

However, this wasn’t enough on its own. I didn’t know my way around all the classes inside iPhoto. To that end, I used class-dump to piece together the classes and their relations.

Eventually after a few hours1, I came up with this.

app := NSApplication sharedApplication
appCtrl := app delegate
albumMgr := appCtrl archiveDocument albumMgr

"Each roll (album) has an imageRec whose name is what we need."
rolls := albumMgr rollAlbums

"Pull out the mapping from pseudo-album name to roll name."
keys := rolls name
values := rolls imageRec name
rollNames := NSDictionary dictionaryWithObjects:values forKeys:keys

"Write the result to a plist."
rollNames writeToFile:'/tmp/roll_names.xml' atomically:YES

One very interesting feature of F-Script is the loop. Or lack thereof. Did you see the rolls name above? rolls is an array, which doesn’t respond to the names method. So F-Script has magic to say “send the names message to each element of the rolls array”. There’s more on this in the messaging patterns section of the language guide. But it feels rather nice.

The end result is that I do have all the data I need to move my photos into the next system. I can now safely dispose of the broken iMac, much to the relief of the room that’s been harboring it.

As an aside, this whole business of Albums vs Rolls very much feels like an example of technical debt in a system which was rushed. There was no obvious reason to have rolls and albums represented so differently. But instead, they were bolted on in a strange fashion.

1Actual time period is undefined.

Happy Fifth Anniversary, Go

So, Go is five years old. Looking back through version control, my first bit of Go was in May 2012. I’ve been using it as my preferred language for the last year or so. It still feels very pleasant and easy to work with. In no small part that’s due to the excellent tooling. The well written standard library also helps. Go is perhaps the first language where I’ve not had a burning desire to write a URL type because the builtin one is so awful (e.g. Python and Java). Inside Google, it’s been said that if you want to understand a piece of infrastructure, you should look at the Go implementation.

Looking back at that first Go code I wrote (a small tool of about 500 lines), it’s far from embarrassing. Certainly when compared to my efforts in other languages. The code has a few useful types, and methods that act upon them. It’s pretty readable (thanks to gofmt). The one bad thing is that I relied on panic far too heavily, instead of returning errors. The code ran happily for a year or so until the need for it was removed.

Anyway, if you’ve not already done so, try writing your next program in Go. You may be surprised at what a pleasant and productive experience it is.

Go Strings

I’ve been looking at Go recently. It’s a pleasant language, with few surprises. However, I wondered (as always) what the encoding of a string is supposed to be. For example:

  • Python 2 has two types: str, and unicode. Python 3 has sensibly renamed these to bytes and str, respectively.
  • Perl has a magic bit which gets set to state that the string contains characters as opposed to bytes (it’s called the UTF-8 bit, but it means characters).

So how does Go deal with characters in strings? Given that the authors of Go also invented UTF-8, we can hope it’s been thought about.

There are three types to think about.


A slice of bytes.


A (possibly empty) sequence of bytes. Strings are immutable.


A single unicode code point. Produced by characters in single quotes.

There’s no explicit encoding in the above. Nonetheless, there’s an implicit preference for UTF-8:

But this doesn’t help the common case:

package main

import "fmt"

func main() {
  s := "café"
  fmt.Printf("%q has length %d\n", s, len(s))

// "café" has length 5

The unicode/utf8 package can do what’s needed though. This provides functions for, amongst other things, picking runes out of strings.

package main

import (

func main() {
  s := "café"
  fmt.Printf("%q has length %d\n", s, utf8.RuneCountInString((s)))

// "café" has length 4

This is very Go-like. The default is somewhat low-level, but the types and libraries build on top of it. For example, text/scanner provides a nice way of iterating over runes in a UTF-8 input stream.

On a whim, I took a look at the internals of utf8.RuneCountInString(). It’s deceptively simple.

func RuneCountInString(s string) (n int) {
  for _ = range s {

This relies on the spec defining how a string interacts with a for loop: it’s defined as iterating over the UTF-8 codepoints (or runes).

jslint4java 2.0.2

After another long period of having nothing to say, I’ve pushed out an update to jslint4java. Version 2.0.2 doesn’t include any major new features, but does update JSLint to the latest version (2012-02-03) and fix a couple of small bugs here and there.

  • issue 75: Handle BOMs when using the CLI.
  • issue 74: Document the technique for construction of JSLint objects.
  • issue 73: Better examples for maven configuration.
  • issue 72 : Add all formatters to the maven plugin automatically.
  • issue 67 : Fix maven docs.
  • Update to JSLint 2012-02-03.
    • This removes the adsafe, confusion and safe options.
    • This adds the anon option.

One minor point: Now that google code supports git repositories, I’m also pushing the source code there again. Github is still my “primary” but there’s another copy. More copies are good.

I did spend a bit of time testing the CLI interface properly. This isn’t really noteworthy, but it was entertaining for me, and hopefully results in fewer bugs like issue 75.

jslint4java 2.0.0

I’ve finally released jslint4java 2.0.0. It’s now available at code.google.com/p/jslint4java. The main new feature is that it now sports a maven plugin in addition to the ant task.

There is also a breaking change, that’s been inherited from JSLint. The meaning of several options has been inverted. Now, the default is to behave strictly, with options turned off. For example, if you want to turn off JSLint’s checking of whitespace, you now have to specify --white. Previously, this would enable checking of whitespace. See the release notes for details, and please take care when updating.

The maven plugin should behave much like any other maven plugin: you add it to your <build><plugins> section. Here’s an example:


I’d love feedback on how well this works.