(cyrus to dovecot mail server conversion)

Migrating the mail service from a Mac OS X 10.5 (or even 10.4) server is not a straightforward process. This holds particularly true if your mail store, hosted by default in ‘var/spool/imap‘ on your boot volume, is hosted elsewhere. If you can afford the downtime however (and for busy servers you’ll need lots) it is definitely worth the hassle. Less CPU intensive, lower memory use, faster to deal with clients and self-healing databases are but a few of the major advantages.

The big issue is that previous servers used _cyus for mail delivery, but OS X 10.6 server uses the vastly superior dovecot system (also an open source solution), and so your mail database needs converting.

Upgrading a previous OS to Snow Leopard.

If you’re doing an upgrade to Snow Leopard server from a previously installed version of OS X, and your mail store is in the default place on your boot volume, then Apple’s ‘Migration Manager’ will do most of the hard work for you (Page 41 of the Snow Leopard ‘Upgrading and Migrating’ manual). However, there are a couple of issues that may still arise, and it would be worth checking over the Issues section at the bottom of this post.

If however you have the time, and there’s not too much else on the server you need to worry about, a clean install seems to be a superior option.

Starting from fresh.

If you’re going from fresh, then you will need a copy of the spool directory, any sieve scripts, any webmail config, address books used, and your mail server config file (usually found at ‘/etc/imapd.conf’). If your server is a Directory Master then you will also need a backup of the directory. (Less issues with updated configs for a start.)

Step 1 – Back up old config files.

If they are not already on a separate drive, you’ll need to back them up onto one before the move. It’s important to keep permissions in tact when doing this so I would recommend using ‘ditto’ or ‘rsync’ as the method of transit.

If your mail server is busy and, like most, down time is a luxury, using rsync will allow you to copy the majority of the data whilst the server is still running, and simply update it when the server is taken off-line.

Step 2 – Install Snow Leopard!

I’ll leave you in apple’s capable hands for this one! Don’t forget to set up a timeserver and link to your LDAP directory at this point. Test LDAP thoroughly as, if it’s not working, your update will fail and cause you a lot of additional work further on. The server needs to query the directory master for each user’s information – more specifically the user’s GUID.

Step 3 – Updating the mail store.

This is the time consuming bit. You will need to allow time for all your files to copy from your old location to the new. Don’t underestimate this – for 500 ish users, 180GB of data and with all disks internal (not usb / firewire etc) the transfer took over 3 hours.

The update is done using the ‘migrate_mail_data.pl‘ script Apple provides. It lives within ‘/usr/libexec/dovecot/’. If at this point, and i would recommend it, you want to read Apple’s published ‘Upgrading and Migrating’ guide you can find it at:

http://www.apple.com/server/macosx/resources/documentation.html

And you’re looking for page 42!

The migrate_mail_data.pl scrip needs to be run as root. Whether you wish to do this directly or via sudo is entirely up to you. If you are running it directly as the root user I would recommend doing so entirely from shell (log into the GUI via a standard admin user). Using the Finder as the root user often leaves unwanted. fstemp files all over the place!

The command is formatted very simply, as shown below, and it’s important to apply all the flags! NB: Your old mail store will be left untouched. The changes to the database are made ‘in transit’.

sudo /usr/libexec/dovecot/migrate_mail_data.pl —moveMail 0 —cyrusBin “/Volumes/OLD-Server/usr/bin/cyrus/bin” –database “/Volumes/OLD-Server/var/imap” –sourceSpool “/Volumes/OLD-Server/var/ spool/imap” –targetSpool “/var/spool/imap/dovecot/mail”

For resigns which I am yet to fathom, leaving the old cyrusBin on the second, non boot disk, caused problems. The process ran much faster if I copied the cyrusBin data onto the main boot drive. It’s likely to only be small. I also converted the permissions to be read and write for all (chmod 777).

Step 4 – Check all transferred ok.

First thing’s first. Check your logs. These will be found in /Library/Logs/MailDataMigration.

On large LDAP directories you will commonly see errors when looking up the GUID for users. This does not mean that your data has not copied over, just that the new mail store for these has not been properly renamed. If you’re getting these errors read the ‘GUID not transferred’ section under the ‘Issues’ heading below.

You should also check your new file store. Where previously all of the users mail folders will have been named with their shortname, they should now be named with their GUID. If not read the ‘GUID not transferred’ section under the ‘Issues’ heading below. This problem is often also flagged up by some users appearing to have a blank, virgin, mailbox.

It’s also worth checking that the number of files in the directory is the same, and that there’s not too much difference in the overall file size.

Step 5 – Configure the Mail Server.

Best to do this through the Apple GUI. I’m going to assume that you’re ok to find your way round this. Things to check for are that the mail store and database pointers are going to the right place and that your SSL settings are the same. Relay servers for incoming and outgoing mail need to be set at this point to. The GUI is very similar to the previous OS X 10.5 version. I take screen shots of mine before starting a migration.

Any sieve scripts your were running should also be transferred at this point.

Your imapd.conf file should also be updated now. It might be worth doing this by hand rather than a straight replacement, as syntaxes my have changed.

Step 6 – Start your server.

That’s it, you’re done! If you’re sure everything copied ok, you can start the service. Clients should not need to make any changes to their configuration, or rebuild their mailboxes.

Issues.

Errors reading the cyrusBin -

This problem is resolved by copying the cyrus/bin folder to the boot drive, and chmod’ing the permissions to 777. This also vastly increases the speed of transfer.

GUID not transferred -

Unfortunately this is not only a common problem, but also a painful process to fix. If you have a lot of cases where the user’s GUID number was not brought through it would be worth scripting the following process. Alternatively get in touch if you would like this done for you.

If this has happened for all users it means your server is not properly configured to talk to your LDAP directory.

You need to manually look up the GeneratedUID number for each user where the mail directory (in /var/spool/imap/dovecot/mail by default) is still named with their ‘shortname’ from the LDAP directory and rename their mail directory with the user’s GUID. To look this up from the command line you can use the command:

dscl /Search read Users/USERNAME GeneratedUID

‘chaps’ is a useful command to test your connection with LDAP.

Clients not reconnecting –

I nearly always forget to enable non SSL connections. – I’d start there.

Need More Help?

Everything above comes from my experience of migrating Apple mail servers from OS X 10.5 to OS X 10.6, but I’m sure there are others! If you find yourself looking for further help, or remote support, please get in touch, and I would be happy to discus some options.

For further free support I regularly find the guys at afp548.com forum invaluable!