Tag: perl


Perl List Slice Weirdness

A colleague at work just found this little gem in File::Copy::Recursive:

  my $two = join '-', ( stat $cur )[0,1] || '';

That should pick out the device and inode, join them with a hyphen and set $two to the empty string if the stat failed. However, there’s a precedence problem:

  # Intended code.
  my $two = join( '-', ( stat $cur )[0,1] ) || '';
  # Actual code.
  my $two = join '-', ( ( stat $cur )[0,1] ) || '';

Spot the difference? The || bit applies to the result of the list slice. But what happens when you use || on a list slice? Let’s look at the debugger…

    DB<1> x @statinfo = stat '/etc/hosts'
  0  2050
  1  327794
  2  33188
  3  1
  4  0
  5  0
  6  0
  7  332
  8  1089379886
  9  1134738486
  10  1134738486
  11  4096
  12  8
    DB<2> x @statinfo[0,1]
  0  2050
  1  327794
    DB<3> x @statinfo[0,1] || ''
  0  327794
    DB<4> x @statinfo[0,1,2] || ''
  0  33188

So it appears that putting a list slice into scalar context always returns the last element of the list. Weird. More info (including examples at perldoc/C-style Logical Or).

Hopefully, he’ll file a bug report as it was actually tripping him up…

Update: RT#19205

Java Leads To Great Libraries

Tim Bray wrote Having Done Java about how writing Java leads to better quality libraries. Needless to say, I disagree. I originally sent this by mail to Tim Bray, but I don’t think that there’s any reason to not publish it here as well.

I’m afraid that I’m going to have to disagree with you on this one. I’ve relatively little experience with Java, but I have been called upon to do some web things. And the standard library makes very, very angry when compared to CPAN[1]. There are a couple of aspects to this:

  • Firstly, everything seems to have been vastly overengineered. Simplicity got thrown out of the window from what I can see. I blogged about one incident (PUTting an URL), but there have been others, frequently involving the bizarre servlets API.
  • Secondly, the strong type system leads to weirdness. This isn’t necessarily Java’s fault, more the people who use it seeing no other way to do things. Again, I’m forced to compare with Perl here. I’m
    thinking in particular of URI.pm, which happily handles https, and http URLs. However, they’re completely different classes in Java. And if you try to feed in one to a method which expects the other, it’s no-go.

Ok, I’ll give in. I’m less moaning about the quality as to the bizarre design which I perceive. But it certainly feels like it should be a factor in any discussion of quality (nebulous as that might be).

1 I’ll admit, however, that there is a huge amount of crap in CPAN too. Anybody remember Meta.pm?

london.pm tech meet

I went to the London.pm tech meet last night. There was an excellent selection of talks, but the one that really caught my eye was Tom Hukins talk on Aspect.pm. Now, I’ve looked at AOP before, mostly in the context of Java and AspectJ.

But every time I read about AOP, I came away with my head spinning. The idea should be simple: provide a way to interleave your own code into other methods that you don’t necessarily control. But the frameworks in existence just don’t seem to be that simple.

So, Tom’s talk, Tracing code with Aspect.pm was really useful. It clearly and simply ushowed how to use Aspect.pm to intercept function calls in a nice, practical manner. You can use it to insert tracing into code that you might not control. What’s very nice is that you can also modify parameters on the way in and out. Tom demonstrated how to redirect all HTTP calls to localhost for testing.

Funnily enough, I first came across this behaviour some years ago inside Emacs: Advising Emacs Lisp Functions. It’s quite simple to use…

Solaris Runtime Linker Fun

Most of the last week at work has been spent trying to coerce my way around the Solaris runtime linker (ld.so.1). The original task is to install a bunch of stuff like OpenSSL, PostgreSQL and Perl into a directory structure ($prefix) on the box. But where the task involves shared libraries, things get tricky…

Compiling and installing the shared library itself is easy. Compiling and installing a dependency on that shared library is just as simple. You have to specify that the include files and libraries are in the right place. You have to do this to get the program to compile, so it gets done anyway, usually by supplying CFLAGS=-I$prefix/include and LDFLAGS=-L$prefix/lib.

Where it starts to get hard is when you run the program afterwards. Because under Solaris, the program doesn’t remember where it found the shared library. So it goes off hunting in a number places, none of which are the ones you want, and then gives up.

The answer is to add in -R$prefix/lib to LDFLAGS. That records the correct place for the shared libraries in the program, and then everything works as expected.

Needless to say, this is exactly what Linux does by default.

Anyway, the end result is that I’ve spent far longer inside the Solaris Linker and Libraries Guide than I really care for. But I do have some working code.

Oh, I almost forgot the most irritating thing that happened during all this: teaching Perl to not look in /usr/local for includes and libraries. There were a few things in there (e.g. gdbm) that I didn’t want to link Perl to because they wouldn’t be on the target machine. But teaching Perl to exclude stuff from its Configure routine is byzantine to say the least. I ended up having to patch the solaris_2.sh hints file, which is (to say the very least) a bit sucky.

Here’s hoping next week will be looking up a bit.

Perl & Ajax

Well, my article on Using Ajax from Perl has finally been published. It’s about using CGI::Ajax and how it makes it really, really simple to add some of that Ajax magic to your own scripts. I’d love to have a look at HTML::Prototype next, as I’ve been looking quite jealously at all the lovely Prototype features. But I need to have a look at how it integrates into Perl.

Perl Improvements

I’m very pleased to see that Nicholas Clark has been given a grant for improving things in Perl. The stuff is useful and should bring big benefits to all Perl users.

The one thing that really catches my eye is “making Perl relocatable”. This is a large annoyance for us at work, where we have multiple Perl installations that are ever-rooted in the same directory. Because they live in directories that are versioned after their projects, it requires that when we perform a new release of our software, we have to rebuild Perl, and all its dependencies. This is irritating, although thanks to Moore’s Law, not too bad these days. But a relocatable Perl will make this a lot simpler.

PUTting an URL

Today, somebody at work wanted to PUT some data to an URL. It’s part of an internal web service we use. Should be easy write. In Perl, it’s pretty trivial.

my $ua = LWP::UserAgent->new();
my $req = PUT 'http://example.com/dest', Content => $ARGV[0];
my $resp = $ua->request( $req );
die $resp->status_line, "\n" unless $resp->is_success;

That takes the first command line argument and PUTs it http://example.com/dest. Pretty simple stuff. But compare this to the Java way, which is what my colleague was attempting.

public class HttpPutTest {
    public static void main(String[] args) throws IOException {
        URL dest = new URL("http://example.com/dest");
        HttpURLConnection conn = (HttpURLConnection) dest.openConnection();
        if (conn.getResponseCode() < 200 || conn.getResponseCode() > 299)
            throw new IOException(conn.getResponseMessage());

Firstly, as with all Java, the sheer verbosity is staggering. But there are two things of interest to me here.

  1. The request and response are conflated into one object, an UrlConnection (or variation thereof). There’s no way to retrieve header values on the request once they’ve been set.
  2. Setting the content is done through the most contorted route. Firstly, we have to tell the connection that it’s to expect output (why? I’m doing output things below, it should work it out). Secondly, we have to fetch an OutputStream, marshal our input into bytes and then write those bytes to the stream.

It’s that OutputStream that gets me. Oh, I can see why you’d need it occasionally. But 90% of the time when you’re doing a PUT or a POST, the amount of data is tiny, and you have it all up front. So you don’t need to stream it to the server. You’re paying the price for it 100% of the time, even though it’s only needed 1% of the time, if that. Really, how hard would a setContent() method be?

XML::Genx 0.20

I pushed out a new release of XML::Genx last night. Nothing worth upgrading for, it just fixes a few compiler warnings under gcc 4.0. Thankfully, after some sage advice from Daniel Jalkut, I managed to get rid of the warnings I was complaining about. Unfortunately, there’s still one warning coming out. But it’s in code that is entirely generated by xsubpp, so there’s not a lot that I can do about it. Ah well.

XML::Genx Compiler Warnings

Naturally, one of the first things I did when I got the new mac was to try compiling my software, XML::Genx, on it. Sadly, it throws up a number of errors.

Genx.xs: In function 'string_sender_write':
Genx.xs:209: warning: pointer targets in passing argument 3 of 'Perl_sv_catpv' differ in signedness
Genx.xs: In function 'string_sender_write_bounded':
Genx.xs:223: warning: pointer targets in passing argument 3 of 'Perl_sv_catpvn_flags' differ in signedness
Genx.xs: In function 'XS_XML__Genx_ScrubText':
Genx.xs:606: warning: pointer targets in passing argument 2 of 'genxScrubText' differ in signedness
Genx.xs:606: warning: pointer targets in passing argument 3 of 'genxScrubText' differ in signedness
Genx.c: In function 'XS_XML__Genx__Namespace_GetNamespacePrefix':
Genx.c:1068: warning: pointer targets in passing argument 3 of 'Perl_sv_setpv' differ in signedness

Looking around the web, it appears that this is a new warning in gcc 4.0. It also baffles me that that apple would have considered using signed characters for anything by default. However, the obvious fix doesn’t work.

-        sv_catpv( *svp, s );
+        sv_catpv( *svp, (signed char *)s );

At this point, I suspect that my knowledge of C is fundamentally lacking. Does anybody out there have any ideas what I need to do to fix this warning? The source is string_sender_write() in Genx.xs, although you’ll probably also want genx.h to look for the typedefs.

Sadly, the gcc manual doesn’t have much to say, except to note the existence of -Wno-pointer-sign to disable the warning. But I’d rather fix it if I can.

Last 100 lines in a file

Yesterday, I had to pick out the last 100 lines in a file, for a small log viewer I was writing. Neither the Perl FAQ nor the Perl Cookbook had quite the right answer. So, this is what I ended up with.

sub last_lines {
  my ( $fh, $howmany ) = @_;
  my ( @lines, $i );
  while ( <$fh> ) {
    $lines[ $i++ ] = $_;
    $i = 0 if $i == $howmany;
  return grep { defined }
    @lines[ $i .. $howmany - 1 ], @lines[ 0 .. $i - 1 ];

This is reasonably memory efficient, more so than the example I found which read the entire file in and then just picked the last $howmany items from the array.

The last line is interesting. We have to “swivel” the array around a partition point, in order to get the entries into the correct order. Also, we have to get rid of undefined entries. These crop up if you have less than $howmany lines in the file.