Postfix 2.5.1 TLS on FreeBSD

This is one of those things that I have to put up there in case anybody else has the same obscure setup that I do…

I run postfix on FreeBSD, using the ports system. This means I have a tendency to just use portupgrade to upgrade to the latest version of anything I happen to have installed. Normally, this works just fine. I usually check the output to see if any warnings about upgrading pop out and that’s about it. Slightly seat-of-the-pants, I know.

Anyway, I recently upgraded to postfix 2.5.1 and started seeing these messages in the logs.

Jul 26 21:29:44 gimli postfix/tlsmgr[7789]: fatal: tls_prng_exch_open: cannot open PRNG exchange file /var/lib/postfix/prng_exch: Permission denied

tlsmgr is the bit of postfix that handles SMTP over SSL.

The first port of call is to look through the postfix release notes. This seemed relevant.

[Incompat 20071206] The tlsmgr(8) and verify(8) servers no longer use root privileges when opening the address_verify_map, *_tls_session_cache_database, and tls_random_exchange_name cache files. This avoids a potential security loophole where the ownership of a file (or directory) does not match the trust level of the content of that file (or directory).

So, what’s the problem?

  % sudo -u postfix ls -l /var/lib/postfix
  ls: /var/lib/postfix: Permission denied
  % sudo -u postfix ls -l /var/lib
  total 0
  ls: lib: Permission denied
  % sudo -u postfix ls -ld /var/lib
  drwxr-x---  5 root  wheel  512 26 Jul 08:14 /var/lib

So, it’s basically a permissions problem. Postfix can’t see the directory it’s trying to use. Previously it wasn’t a problem, as postfix was doing things as root, and root sidesteps permissions checks.

What to do? The simplest is to change the permissions. But I don’t particularly like doing that on systems directories, as they may well get reset in the future (e.g. nightly runs of mtree). So the simplest option is probably to reconfigure postfix to use a different directory. One that it actually has permission to access, like /var/db/postfix.

Annoyingly, when I look at the port to understand this problem (PR#121236), it was fixed in April. I wonder why I didn’t get the fix?

As it turns out a reinstall of postfix (portupgrade -f postfix-2.5.1_2,1) completely fixes the problem, and the directory it uses is now /var/db/postfix by default. I wonder what caused it to go wrong in the first place though?


cfengine Introduction

For years, I’ve looked at cfengine and thought “I must get around to learning that”. Today, I was reading Building a Self-Healing Network and finally thought that I need to give it a try.

One of my problems is that processes on my FreeBSD server often get stopped when I run portupgrade and I don’t notice. cfengine claims to be good at fixing non-running processes, so I gave it a try and came up with this:

      actionsequence = ( processes )

      "amavisd"        restart "/usr/local/etc/rc.d/amavisd restart"
      "clamd"          restart "/usr/local/etc/rc.d/clamav-clamd restart"
      "dhcpd"          restart "/usr/local/etc/rc.d/ restart"
      "dovecot"        restart "/usr/local/etc/rc.d/dovecot restart"
      "freshclam"      restart "/usr/local/etc/rc.d/clamav-freshclam restart"
      "httpd"          restart "/usr/local/etc/rc.d/apache restart"
      "named"          restart "/etc/rc.d/named restart"
      "nmbd"           restart "/usr/local/etc/rc.d/samba restart"
      "ntpd"           restart "/etc/rc.d/ntpd restart"
      "postfix/master" restart "/usr/local/etc/rc.d/postfix restart"
      "postmaster"     restart "/usr/local/etc/rc.d/ restart"
      "smartd"         restart "/usr/local/etc/rc.d/smartd restart"
      "smbd"           restart "/usr/local/etc/rc.d/samba restart"
      "squid"          restart "/usr/local/etc/rc.d/squid restart"

When run via cfagent -f checkprocs.conf, this works a treat.

The next step should be getting all the other cfengine gumph working on my network. However, I’ve only got one box, so running it every hour out of cron is good enough.

  0 * * * * root /usr/local/sbin/cfexecd -f /usr/local/etc/cfengine/checkprocs.conf

In accordance with the cfexecd documentation, I also amended my control section so I’ll get mail when it does something.

      actionsequence = ( processes )
      smtpserver = ( localhost )
      sysadm = ( )

This seems to be a good start on getting cfengine working. Now I can play more with it. It seems like it’d be another useful tool for work.

Update: cfexecd doesn’t work. Running cfagent directly does. How annoying.


Server Move

Well, after a couple of very long and frustrating evenings, I’ve finally managed to replicate the setup of my old, decrepit server onto a shiny new (and hopefully more reliable) dell box. I thought that this would be easy, as I was under the impression that I had everything under version control. But it’s only when you start on an endeavour like this that you realise just how much tweaking you have done to a box. Some particular highlights of this move:

  • I had been installing my squid.conf to the wrong place for about 2 years.
  • I discovered I’d been running clamscan (command line) instead of calling clamd (daemon) for a year. Slow mail ‘r’ us.
  • I discovered that my PostgreSQL backups had not been running for over a year.
  • I noticed that my trac installation had been wikispammed a few days ago (I didn’t notice when it happened). Trac needs a “revert to previous version” button.
  • Rails has caused me the most grief, surprisingly. getting typo up and running has been a pain. Firstly, because I didn’t have FastCGI installed. But I didn’t get an error for some reason. But whe I did install it, all I got was “FastCGI: incomplete headers (35 bytes) received from server”. I have now learned that this means “something went wrong but I’m not going to tell you what is. In my case, it was needing to install the ruby-postgres drivers. But rails isn’t kind enough to tell you that it can’t load the db drivers you’ve requested, instead it just silently fails. Which is a bit of a nuisance. I’ll have to check the trunk to see if it’s been fixed…

Anyway, now I can finally get on with the other more important tasks that I have to do. I gave up being a sysadmin 5 years ago for a damned good reason.


New Server

So I went ahead and bought a new Dell PowerEdge SC430, as my normal server is falling over on a far too regular basis. Lovely. Except that when I got it home, I made the discovery that Dell no longer fit PS/2 keyboard connectors. There are 7 USB slots, however. And guess which kind of keyboard I don’t posess?

On top of that, I also found that the only free-standing monitor I posess only had a DVI cable, not a VGA.

So, back to work to liberate some parts from the stores for an evening. And now I have a very nice shiny new server running FreeBSD.

Now starts the hard bit, configuring it the same as the other one…


FreeBSD / mod_fastcgi

Eventually, I managed to get mod_fastcgi to compile on FreeBSD. See PR#91190. It basically amounted to applying the patch in the mailing list automatically. But what really surprised me were the changes to the port’s Makefile. I trimmed out about 50 lines of confusion into this:

SRC_FILE=    *.c
PORTDOCS=    LICENSE.TERMS mod_fastcgi.html

This is all thanks to the new support. It really works well. However, it is slightly underdocumented. But this isn’t much of a problem. less /usr/ports/www/mod_*/Makefile provided enough hints to get going quickly.

Now I just need to get the maintainer approval, in order to get this committed…


Bloody Software

Over Christmas, my server has mostly been turned off. This is because it’s in one of the guest bedrooms and my mother who was staying in there didn’t appreciate the noise.

On the day after boxing day, we travelled up to the in-law’s for a few days. And so I switched the server on and ran out of the door. Unfortunately PostgreSQL failed to start up correctly (a recent change to the FreeBSD ports system, to which PostgreSQL had not been adjusted). Thanks to the marvels of PHP, this blog just displayed a database connect error. Including the database username and password in the clear. How screwed is that?

What’s worse is that I kept getting “connection refused” when attempting to connect from the in-law’s broadband connection. I have no idea why, but I suspect that some filtering is occuring in their router or at BT Internet. Grrr. So no way to fix it until I got back, just now.

So, the sooner I can get Typo up and running instead, the better. It won’t work without a database, but I very much doubt that it will default to spamming my username and password to the world… Good job I don’t use that one elsewhere.


FreeBSD Crash Dump Analysis

Unfortunately, I had a crash earlier on my FreeBSD server. It’s probably due to bad hardware. But I decided to take a look at the dumpfile anyway. The best resource is the FreeBSD handbook section on Debugging a Kernel Crash Dump with kgdb. That’s invaluable.

Sadly, it didn’t tell me much:

Fatal trap 12: page fault while in kernel mode
fault virtual address   = 0x8
fault code              = supervisor write, page not present
instruction pointer     = 0x20:0xc04c8f79
stack pointer           = 0x28:0xd13ddb44
frame pointer           = 0x28:0xd13ddb54
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, def32 1, gran 1
processor eflags        = resume, IOPL = 0
current process         = 864 (squid)
trap number             = 12
panic: page fault

So at the time, squid was sitting around waiting for things to happen (the stacktrace showed it as being inside poll()) and the machine blew up. Darn. Ah well, the standard response is to just make world and try again.


FreeBSD Perl Upgrade

Like quite a few people, I dread upgrading Perl on my FreeBSD box. It’s a lot of hassle because it involves rebuilding a large number of dependent ports to use the new Perl. So I was pleased to see this blog entry stating how easy it was with the right tools. So here we go, let’s see how easy it can be…


Easier Porting

I’ve been updating all my FreeBSD ports recently and one continuing pain is the gerenation fo the plist file. Looking the Porters Handbook gave me the answer though. There’s a script you can use to make a plist. It needs a bit of editing afterwards, but it’s a good start.

The only problem is that you have to install the port to a temporary area first. This appears to be the best way to do it as an ordinary user instead of root.

% make install PREFIX=/tmp/myport INSTALL_AS_USER=yes NO_PKG_REGISTER=yes

After that, just run the magic script.

/usr/ports/Tools/scripts/plist -Md -m /etc/mtree/BSD.local.dist /tmp/myport > pkg-plist