Rescuing Data with F-Script

I have an old iMac. It’s not in a good state.

Unwell iMac

I don’t care that much, it’s old slow, and not a lot of use. The only remaining thing of value is the large collection of photos I have on there. I have an external drive, so it’s easy to copy them over.

All the photos are stored in iPhoto 5, which (from googling around) appears to be the version before they moved all the metadata into a big binary blob. And looking in the iPhoto Library folder revealed a file AlbumData.xml which contained details of all 12,000+ photos. Result!

Apart from one slight problem. I’d never used iPhoto’s “albums.” Well, I’d made one or two (mostly by accident). But for the most part, I’d left the photos in the import “rolls”, just renaming (e.g.) “Roll 42” to “Trip to Wales”. Now the rolls are represented in AlbumData.xml, but without the roll names. Just “Roll 1”, “Roll 2”, etc. <facepalm/>

So like any good engineer, I’ll spend a huge amount of time to avoid having to re-enter 340 roll names, which I could have done in a couple of hours. This information is necessary for whatever system I’ll be pulling the photos into afterwards, dammit!

iPhoto obviously knows what the name of each roll is, or it wouldn’t be able to display them. They’re just stored in some different place (a binary blob, as it happens). Thankfully, I can still run iPhoto. So long as I avoid the middle of the screen, anyway. Now I forget exactly where I heard about it (Core Intuition, perhaps?) but I remembered that F-Script Anywhere allowed inspecting a running Cocoa program.

F-Script itself is a smalltalk based language, built on top of Objective-C and the Cocoa runtime. The tutorial includes a nice side-by-side comparison with Objective-C. It’s pretty simple for the most part. F-Script Anywhere is an add-on that allows you to inject the F-Script runtime into any running process. I tried it with iPhoto and immediately had a command line and browser with complete access to all of iPhoto. It helped having a minimal knowledge of Cocoa (I could find the app delegate!)

However, this wasn’t enough on its own. I didn’t know my way around all the classes inside iPhoto. To that end, I used class-dump to piece together the classes and their relations.

Eventually after a few hours1, I came up with this.

app := NSApplication sharedApplication
appCtrl := app delegate
albumMgr := appCtrl archiveDocument albumMgr

"Each roll (album) has an imageRec whose name is what we need."
rolls := albumMgr rollAlbums

"Pull out the mapping from pseudo-album name to roll name."
keys := rolls name
values := rolls imageRec name
rollNames := NSDictionary dictionaryWithObjects:values forKeys:keys

"Write the result to a plist."
rollNames writeToFile:'/tmp/roll_names.xml' atomically:YES

One very interesting feature of F-Script is the loop. Or lack thereof. Did you see the rolls name above? rolls is an array, which doesn’t respond to the names method. So F-Script has magic to say “send the names message to each element of the rolls array”. There’s more on this in the messaging patterns section of the language guide. But it feels rather nice.

The end result is that I do have all the data I need to move my photos into the next system. I can now safely dispose of the broken iMac, much to the relief of the room that’s been harboring it.

As an aside, this whole business of Albums vs Rolls very much feels like an example of technical debt in a system which was rushed. There was no obvious reason to have rolls and albums represented so differently. But instead, they were bolted on in a strange fashion.

1Actual time period is undefined.


SSHKeyChain install fix

I like SSHKeychain. I’ve used it for a long time. However, I recently tried to install it and noticed that it kept blowing up in the post-install phase. Having quick look at the installer, I saw this:

% cd /Volumes/SSHKeychain 
% cat SSHKeychain.pkg/Contents/Resources/postinstall 
chown -R $USER:$USER "$2/"
#chown root:admin "$2/"
#chmod u+s "$2/"

It assumes you have a group named after you. This is not the case for me. Now, I could copy the package to a read-write location, and edit the postinstall script to fix this.

However, it turns out the failure in postinstall means the package still installed OK. So, it’s easy to fix the failure by manually chown(1)ing at the command line.

% sudo chown -R $LOGNAME /Applications/

Xcode Build Settings

One thing I keep coming up against in Xcode projects is that the build settings aren’t quite right. Why is ZERO_LINK enabled when target T2 is built with configuration C4? Unfortunately, you quickly end up with a very large combination of options. Six targets and four configurations means twenty four combinations to think about. Which is madness.

There is an easier way: build settings files (*.xcconfig). These are text files with the same settings that are normally embedded into your xcodeproj file. For example, on a newly created cocoa application, these are the settings that are present by default in the Debug configuration.


And these are the ones in the Release configuration.


Hmmm, they look quite similar (run diff(1) to see). We can extract a common file (Choose File → New File… → Other → Configuration Setting File).

// Project-Common.xcconfig


// Project-Debug.xcconfig

#include "Project-Common.xcconfig"


// Project-Release.xcconfig

#include "Project-Common.xcconfig"

To apply these, go into the Build tab of the Project Info pane, and select the Debug configuration. Delete all the settings in there (⌘-A, ⌫) and then select “Based On: Project-Debug”. Do the same for the Release configuration, except base it on Project-Release.

Debug configuration settings

Now, you can add a setting to Project-Common.xcconfig and it will show up in all your configurations. As a nice side effect, it’ll be much easier to see the change in version control.

What about targets? Well, the same principle applies. Except that you want to “push up” as much configuration as possible into your project level xcconfig files.

Let’s take a look at the default Debug configuration for my “Polynomials” target.

INSTALL_PATH = $(HOME)/Applications
PRODUCT_NAME = Polynomials
GCC_PREFIX_HEADER = Polynomials_Prefix.pch

The only really target specific settings are INFOPLIST_FILE and PRODUCT_NAME. The rest can either be deleted (GCC_MODEL_TUNING?!?), or pushed up into the project level settings (GCC_OPTIMIZATION_LEVEL, which is already defined there).

It’s a really worthwhile exercise going through your project in order to clean up your build settings. You’d be surprised at what’s going on in there. Keep a copy of the Xcode Build Settings Reference handy to look things up and see if you really need them. If you want an example, I noticed that Eric Czarny’s lovely XMLRPC library is organised somewhat like this.

You can do a lot more mad things with xcconfig’s, like choosing alternate signing personalities. But extracting your current configuration is a good start.

In order to help the transition, I wrote a bit of applescript to pull out existing settings. It’s pretty nasty because I don’t really get applescript. But it Works For Me™.

set newLine to ASCII character 10

tell application "Xcode"
    set proj to project 1
    -- This is a string of a POSIX path.
    set dir to project directory of proj

    repeat with cf in build configurations of proj
        set str to ""
        repeat with stg in build settings of cf
            set str to str & (name of stg) & "=" & (value of stg) & newLine
        end repeat

        -- Write the results to a file
        set filename to "Project-" & (name of cf) & ".xcconfig"
        set fileno to open for access (dir & "/" & filename) as POSIX file with write permission
        write str to fileno
        close access fileno

        -- Sort the file to make it easier to diff.
        do shell script "cd " & dir & " && sort " & quoted form of filename & " >" & quoted form of filename & ".tmp && mv " & quoted form of filename & ".tmp " & quoted form of filename
    end repeat

    -- Now, iterate through each target.
    repeat with tgt in targets of proj
        repeat with cf in build configurations of tgt
            set str to ""
            repeat with stg in build settings of build configuration (name of cf) of tgt
                set str to str & (name of stg) & "=" & (value of stg) & newLine
            end repeat

            -- Write the results to a file
            set filename to "Target-" & (name of tgt) & "-" & (name of cf) & ".xcconfig"
            set fileno to open for access (dir & "/" & filename) as POSIX file with write permission
            write str to fileno
            close access fileno

            -- Sort the file to make it easier to diff.
            do shell script "cd " & dir & " && sort " & quoted form of filename & " >" & quoted form of filename & ".tmp && mv " & quoted form of filename & ".tmp " & quoted form of filename
        end repeat
    end repeat
end tell

And yes — applescript doesn’t have a sort function builtin. WTF?



I’m working on an OSX interface to a web service right now. Of course, one of the requirements is for a username and password. Simple — just use keychain, right?

Well, it might be simple, apart from the fact that Keychain Services has a hoary old API. It’s C-based, and is like taking a step back in time to the early ’90s. After using the rest of the Cocoa Objective-C APIs, it’s a real shock.

  // Try to find a password.
  const char *svc = "ABCD";
  SecKeychainItemRef itemRef = nil;
  OSStatus st = SecKeychainFindGenericPassword(NULL, strlen(svc), svc, 0, NULL,
                                            NULL, NULL, &itemRef);

  if (st == noErr) {
    // use the keychain item
  } else {
    NSLog(@"Ooops, got error status %d", st);

Pulling items out of a keychain is even more long winded.

  UInt32 length;
  char *password;
  SecKeychainAttribute attributes[4];
  SecKeychainAttributeList list;
  // list attributes we want to read.
  attributes[0].tag = kSecAccountItemAttr;
  attributes[1].tag = kSecDescriptionItemAttr;
  attributes[2].tag = kSecLabelItemAttr;
  attributes[3].tag = kSecModDateItemAttr;

  list.count = 4;
  list.attr = attributes;
  // Can pass NULL for password if not needed.
  OSStatus status = SecKeychainItemCopyContent(item, NULL, &list, &length, (void **)&password);

  if (status == noErr) {
    // Use the now filled-in attributes.  NB: Password is NOT zero terminated, but "length" long.
  } else {
    // Complain bitterly.
  SecKeychainItemFreeContent(&list, password);

So what to do? Of course, it’d be great if there were a nice, OO wrapper. I had a quick look and found EMKeyChain (on github). Here’s a sample usage:

  NSString *password = [EMGenericKeychain passwordForUsername:@"dom"

It’s a world of difference. Sadly, EMKeychain doesn’t quite support the use case I want. I don’t necessarily know the username in advance. This leaves me with a few choices:

  1. Give up, and just use the vanilla Keychain API. Bleurgh.
  2. Fix EMKeychain to support my use case.
  3. Use the preferences API to store the username, and then EMKeychain will work as expected.

I’m leaning quite strongly towards the third option.


The joy of apple keyboards

Recently, I’ve been using a Linux desktop for the first time in ages. It’s Ubuntu (Hardy Heron), and it looks nice. But after using a mac for three years, I’m really missing quite a few little things.

  1. The ability to drag and drop anything anywhere.
  2. Being able to type a wide range of Unicode characters easily.

On a mac, it’s really, really easy to type in a wide variety of useful characters. All you need is alt (⌥), sometimes known as “option”.

Keys Character Name
⌥ ⇧ - EM DASH

How did I find all this out? The lovely keyboard viewer that comes with OS X. You can get the flag in your menu bar by going to International in system preferences and checking “Show input menu in menu bar.”

Selecting the keyboard viewer in the input menu
OS X Keyboard Viewer (normal state)

Now, hold down alt and see what you can get (try alt and shift too).

OS X Keyboard Viewer (alt)

But not everything is attached to a key. In case you need more characters, there’s always the character palette. Usually on the ⌥ ⌘ T key as well as in the Edit menu. Here, you can get access to the vast repertoire of characters in Unicode. Need an arrow?

Arrows in the Character Palette

There’s a lot you can do with the character palette, but the search box is probably the best way in. Just tap in a bit of the name of the character you’re looking for and see what turns up.

This easy access to a wide array of characters is something I’ve rather come to take for granted in OS X. So coming back to the Linux desktop, it was odd to find that I couldn’t as readily type them in. Of course, I haven’t invested the time in figuring out how to set up XKB correctly. Doubtless I could achieve many of the same things. But my past experiences of XKB and it’s documentation have shown me how complicated it can be, so I don’t rate my ability to pull it off.

The end result is that I’m spending most of my time on the (mac) laptop and ignoring the desktop. I do like my characters. 🙂


ASL (Apple System Log)

I’ve just been debugging a problem with the pulse-agent on a mac. One of the big questions we had was: where the heck are the logs? The pulse-agent is managed by launchd. Apparently, this logs all stdout and stderr via ASL.

But what’s ASL? Apparently, it’s the Apple System Log. There’s a nice summary on Domain of the Bored. He also gives the key hint: you can use syslog(1) to read the binary ASL files.

I didn’t delve too deeply into the flags. It appeared that just running syslog spat out all the information I required. But, it comes out encoded like cat -v. But you can pipe it through unvis(1) to clean that up.

$ syslog | unvis | less

Normally, would take care of all this transparently. But when you’re ssh’d into a mac, that’s not an option. So it’s good to know about syslog(1).

Looking closer at the flags, you can say syslog -E safe in place of piping through unvis(1).

Uncategorized on OSX

Just a quick note… I was looking at RT#48699 when I noticed that MacPorts didn’t have in it’s collection. I needed to install it by hand. Unfortunately, the latest version (1.12) doesn’t install cleanly.

So I’ve forked it and fixed it (along with a couple of other minor nits).

Claes said he’ll apply the patch at some point. So hopefully when 1.13 comes out, this won’t be necessary.

Of course, really I should get to grips with MacPorts and submit a Portfile


Using a Java 6 based Eclipse with Cocoa

This is somewhat niche, but I’m going to post it anyway in case it helps somebody else

I recently saw a problem with Eclipse and m2eclipse. When I tried to import a Java 6 based project, I got an error in the maven console.

Failure executing javac, but could not parse the error:
javac: invalid target release: 1.6
Usage: javac  
where possible options include:
 -g                         Generate all debugging info
 -g:none                    Generate no debugging info
 -g:{lines,vars,source}     Generate only some debugging info
 -nowarn                    Generate no warnings
 -verbose                   Output messages about what the compiler is doing

This is happening because m2eclipse is trying to run the compiler from within the same JVM that Eclipse is running. And by the downloads only offer Carbon and Cocoa options. Both of these are 32 bit. Which means they’ll only ever run using Java 5, even if you’ve got Java 6 installed (unlike my iMac G5, grumble, grumble).

Thankfully, the 64 cocoa version is available, though it’s only the old “Eclipse SDK” download. But ekke wrote up [galileo] EPP for Cocoa 64-bit, which shows how to go about getting (effectively) the same setup as the downloads.

If you follow along that procedure, you get an eclipse that works well with m2eclipse and Java 6 project. As a bonus, it feels quicker to me.

Hopefully future versions of Eclipse will offer a 64-bit cocoa download.


Eclipse 3.5 in Cocoa

I’m just trying out Eclipse 3.5-RC4. One of the big new features for me is that it’s now based on Cocoa instead of Carbon. There are many benefits to this, including being able to run on 64-bit Java 6. Fundamentally, it just looks and feels a little bit more mac-like.

As an example, one nice little mac feature I use a bit is “lookup this word in the dictionary”. If you hit Ctrl-Cmd-D and hover over a word, you get an in-place definition. Eclipse now does this:

Looking up a word in the dictionary inside Eclipse

It doesn’t work everywhere (which you can probably guess from the context of that screenshot), but it is an indicator that Eclipse & mac are coming together. This is great news for Java developers on the mac.

Oh, it does feel a little bit faster too, which can’t hurt.



Like a lot of people, most of my Unix knowledge comes from an early reading of Advanced Programming in the UNIX Environment. This is an excellent tome on the interfaces provided by the kernel to programs on a Unix system.

Unfortunately, it’s over 15 years old now, and things have moved on. Naturally, I haven’t quite kept up. So I’ve just been pleasantly surprised to see that OS X has grown a sandbox system (via). There is scant documentation available:

Also, if you poke around, you’ll find /usr/include/sandbox.h and /usr/share/sandbox. The latter is interesting — it contains lisp-like definitions of access control lists for various processes.

What’s interesting to me is sandbox-exec though. This can be used with one of the builtin profiles to easily restrict access. For example:

$ sandbox-exec -n nowrite touch /tmp/foo
touch: /tmp/foo: Operation not permitted

After using strings(1) on apple’s libc (/usr/lib/libSystem.dylib), I managed to get these builtin profile names out:

TCP/IP networking is prohibited
All sockets-based networking is prohibited.
All operating system services are prohibited.
File system writes are prohibited.
File system writes are restricted to the temporary folder /var/tmp and the folder specified by the confstr(3) configuration variable _CS_DARWIN_USER_TEMP_DIR.

They’re only documented as internal constants for C programs, but it’s quite handy to have them available for sandbox-exec. It would be nice to know in more detail what they actually did, though.

Of course, this still isn’t really getting down to how the sandbox is implemented. Is it done inside the kernel or on the userland side? I don’t really know. And I don’t yet have enough dtrace-fu to figure it out.

See also:

Anyway, this seems like a fun toy. And of course, it’s reminded me that I need to try out chromium on the mac… Drat, no PPC support. 😦