These two nifty commands allow you to set separate permissions on files or directories within a document root on POSIX based file systems such as linux or Mac OS X with a single comment. They’re an invaluable tool in any web developers arsenal!
To reset the permissions on all files within a document root you can run from the command:
find . -type f -exec chmod 644 {} \;
Or for all directories (files) within the document root:
find . -type d -exec chmod 755 {} \;
I had a problem with a testing server earlier today where the kerberos database had become corrupt. For any users on my Open Directory Master the kerberos passwords were flagged as incorrect, and changing them from work group manager had no effect. Changing them from the command line was not an option as this relies on knowing the users original password – which was corrupt.
The kerberos service can be restarted by greping the output of the ‘ps aux’ command for the kerberos process, (usually named ‘krd5kdc’) and then issuing a kill command against its PID. The service will then automatically restart.
ps aux | grep krd
kill kerberos_PID
This slightly improved the problem as it allowed newly created users to use kerberos with their correct password.
Re-building the kerberos database is done with the following command:
slapconfig -kerberize -f diradmin
This needs to be run as root, either directly of via sudo. The -f flag forces the current set up to be over written.
I would recommend taking a full backup of your users and groups, as well as an archive of your Open Directory server from server admin. Stopping any services that rely on kerberos would also be a good idea.
If neither of the above options worked then it is possible to rebuild your kerberos database from scratch – nuking your old database. This would also be necessary if you are changing the the Kerberos Domain, however don’t forget that if doing this you would also have to change the search path in all your LDAM and Password Server databases.
To completely rebuild kerberos.
1) Stop the OD Service.
2) Log into a shell as root and run the following command:
sso_util remove -k 0a diradmin -p your_diradmin_password -r your_kerberos_realm
3) Remove the following files and directories from your system:
/var/db/krb5kdv/
/Library/Preferences/edu.mit.Kerberos
/etc/krb5.keytab
4) Run the following set of commands as root:
dscl 127.0.0.1
cd /LDAPv3/127.0.0.1/Config/
auth diradmin (and enter your diradmin password)
delete KerberosKDC
delete KerberosClient
quit
5) find and kerberos process (krb5kdc) and kadmind processed and kill them, (as shown above).
6) Re-build your kerberos database:
slapconfig -kerberize diradmin
You can check if the users’ passwords are now being accepted using the ‘kpasswd’ command. The ‘kinit’ command can also be used to test creating a kerberos ticket.
Every now and again when applying a quota change to a user on a Dovecot mail server the change will fail to be implemented. On a cPanel server in particular this will result in an error message being thrown back to the administrator. On other systems the change will often appear to have been applied, but in reality will have failed.
The result of this will either be that the old quota level remains active, or occasionally that the user’s mail box is permanently flagged as full. Obviously with the latter, this will cause emails being sent to the user being returned to the sender. Often this is the first clue that something has gone wrong.
The fix for this: Simply navigate to the user’s mail directory and delete the maildirsize file.
rm -f maildirsize
The next time the user receives an email the file will be recreated by the Dovecot process and the quoter system will be repaired. Once that’s done you’re able to re-a-sign the new quota level to the user.
Very occasionally it is just that the file gets corrupt. However thanks to Dovecot’s ‘self healing’ abilities this is normally resolved automatically the next time the user receives an email. A more likely cause is that the permissions on the maildirsize file have been altered, preventing the Dovecot process from being able to write to the file.
The most common cause however is that the quota was changed, and then before it was implicated by the server, (which happens on receipt of a new email), the quota was adjusted again. If this happens continuously this problem will arise.
I’ve been asked allot recently what the correct permissions should be for the mail store on Apple’s Snow Leopard (10.6) dovecot mail server, so here’s a list as they appear on the servers I work on:
File / Directory Name | Owner | Group | Permissions (chmod) | |
|---|---|---|---|---|
/var/spool | root | wheel | 755 | (drwxr-xr-x) |
/var/spool/imap | root | wheel | 755 | (drwxr-xr-x) |
/var/spool/imap/dovecot | root | wheel | 755 | (drwxr-xr-x) |
/var/spool/imap/dovecot/mail | _dovecot | 775 | (drwxrwxr-x) | |
Users maildir (as users GUID) | users shortname | 700 | (drwx------) | |
cur | users shortname | 700 | (drwx------) | |
dovecot-keywords | users shortname | 644 | (-rw-r--r--) | |
dovecot-uidlist | users shortname | 600 | (-rw-------) | |
dovecot-uidvalidity | users shortname | 600 | (-rw-------) | |
dovecot.index | users shortname | 600 | (-rw-------) | |
maildirsize | users shortname | 600 | (-rw-------) | |
new | users shortname | 700 | (drwx------) | |
subscriptions | users shortname | 600 | (-rw-------) | |
tmp | users shortname | 700 | (drwx------) |
.fstemp files are the temporary files that Mac OS X creates when synchronising files. They are predominantly used for Home Syncing by network clients and syncing with Apples iDisk product. Normally once a the task / process has finished the .fstemp files will be removed, however if errors occur or the process is interrupted the .fstemp files will be left.
These temporary files can get very large, and often end up consuming large amounts of disk space. – Yet as they are not visible from the finder it’s not immediately apparent where the space has gone. Removing the .fstemp files is often a good place to start regaining that invisible, and illusive, missing disk space!
.fstemp files can be seen by running the ls –la command from the terminal (shell). As the filename starts with a ‘.’ it is a hidden file. To see how many you have lurking on your mac run the following command in a shell. On busy servers this can be huge number:
find . -name ‘.fstemp’ | wc -l
The easiest way to remove them all is to run this command in a shell. Run it as root if needs be:
find . -name ‘.fstemp’ | xargs rm -fr
One of the best ways is to stop background syncing by network clients. By syncing only at time of log out or log in you’re ensuring all applications are closed and nothing will be preventing copying.
I’ve also found that they seem most common when syncing the larger files. Not syncing the users iPhoto library’s or iMovie projects helped on my system.
If you do want to keep using background sync, not synchronizing the users library often helps although this will loose the backup of any preferences they have saved.
I’ve never had any problems from doing this. Best to do it when the servers quiet if possible. On troublesome servers I sometimes add it into the cron for the early hours of the morning.
Removing .fstemp files on OS X can be done by run the following command from a shell (terminal). It will tell you how many there are and then delete them all:
find . -name ‘.fstemp’ | wc -l && find . -name ‘.fstemp’ | xargs rm -fr
(cyrus to dovecot mail server conversion)
Migrating the mail service from a Mac OS X 10.5 (or even 10.4) server is not a straightforward process. This holds particularly true if your mail store, hosted by default in ‘var/spool/imap‘ on your boot volume, is hosted elsewhere. If you can afford the downtime however (and for busy servers you’ll need lots) it is definitely worth the hassle. Less CPU intensive, lower memory use, faster to deal with clients and self-healing databases are but a few of the major advantages.
The big issue is that previous servers used _cyus for mail delivery, but OS X 10.6 server uses the vastly superior dovecot system (also an open source solution), and so your mail database needs converting.
If you’re doing an upgrade to Snow Leopard server from a previously installed version of OS X, and your mail store is in the default place on your boot volume, then Apple’s ‘Migration Manager’ will do most of the hard work for you (Page 41 of the Snow Leopard ‘Upgrading and Migrating’ manual). However, there are a couple of issues that may still arise, and it would be worth checking over the Issues section at the bottom of this post.
If however you have the time, and there’s not too much else on the server you need to worry about, a clean install seems to be a superior option.
If you’re going from fresh, then you will need a copy of the spool directory, any sieve scripts, any webmail config, address books used, and your mail server config file (usually found at ‘/etc/imapd.conf’). If your server is a Directory Master then you will also need a backup of the directory. (Less issues with updated configs for a start.)
Step 1 – Back up old config files.
If they are not already on a separate drive, you’ll need to back them up onto one before the move. It’s important to keep permissions in tact when doing this so I would recommend using ‘ditto’ or ‘rsync’ as the method of transit.
If your mail server is busy and, like most, down time is a luxury, using rsync will allow you to copy the majority of the data whilst the server is still running, and simply update it when the server is taken off-line.
Step 2 – Install Snow Leopard!
I’ll leave you in apple’s capable hands for this one! Don’t forget to set up a timeserver and link to your LDAP directory at this point. Test LDAP thoroughly as, if it’s not working, your update will fail and cause you a lot of additional work further on. The server needs to query the directory master for each user’s information – more specifically the user’s GUID.
Step 3 – Updating the mail store.
This is the time consuming bit. You will need to allow time for all your files to copy from your old location to the new. Don’t underestimate this – for 500 ish users, 180GB of data and with all disks internal (not usb / firewire etc) the transfer took over 3 hours.
The update is done using the ‘migrate_mail_data.pl‘ script Apple provides. It lives within ‘/usr/libexec/dovecot/’. If at this point, and i would recommend it, you want to read Apple’s published ‘Upgrading and Migrating’ guide you can find it at:
http://www.apple.com/server/macosx/resources/documentation.html
And you’re looking for page 42!
The migrate_mail_data.pl scrip needs to be run as root. Whether you wish to do this directly or via sudo is entirely up to you. If you are running it directly as the root user I would recommend doing so entirely from shell (log into the GUI via a standard admin user). Using the Finder as the root user often leaves unwanted. fstemp files all over the place!
The command is formatted very simply, as shown below, and it’s important to apply all the flags! NB: Your old mail store will be left untouched. The changes to the database are made ‘in transit’.
sudo /usr/libexec/dovecot/migrate_mail_data.pl —moveMail 0 —cyrusBin “/Volumes/OLD-Server/usr/bin/cyrus/bin” –database “/Volumes/OLD-Server/var/imap” –sourceSpool “/Volumes/OLD-Server/var/ spool/imap” –targetSpool “/var/spool/imap/dovecot/mail”
For resigns which I am yet to fathom, leaving the old cyrusBin on the second, non boot disk, caused problems. The process ran much faster if I copied the cyrusBin data onto the main boot drive. It’s likely to only be small. I also converted the permissions to be read and write for all (chmod 777).
Step 4 – Check all transferred ok.
First thing’s first. Check your logs. These will be found in /Library/Logs/MailDataMigration.
On large LDAP directories you will commonly see errors when looking up the GUID for users. This does not mean that your data has not copied over, just that the new mail store for these has not been properly renamed. If you’re getting these errors read the ‘GUID not transferred’ section under the ‘Issues’ heading below.
You should also check your new file store. Where previously all of the users mail folders will have been named with their shortname, they should now be named with their GUID. If not read the ‘GUID not transferred’ section under the ‘Issues’ heading below. This problem is often also flagged up by some users appearing to have a blank, virgin, mailbox.
It’s also worth checking that the number of files in the directory is the same, and that there’s not too much difference in the overall file size.
Step 5 – Configure the Mail Server.
Best to do this through the Apple GUI. I’m going to assume that you’re ok to find your way round this. Things to check for are that the mail store and database pointers are going to the right place and that your SSL settings are the same. Relay servers for incoming and outgoing mail need to be set at this point to. The GUI is very similar to the previous OS X 10.5 version. I take screen shots of mine before starting a migration.
Any sieve scripts your were running should also be transferred at this point.
Your imapd.conf file should also be updated now. It might be worth doing this by hand rather than a straight replacement, as syntaxes my have changed.
Step 6 – Start your server.
That’s it, you’re done! If you’re sure everything copied ok, you can start the service. Clients should not need to make any changes to their configuration, or rebuild their mailboxes.
Errors reading the cyrusBin -
This problem is resolved by copying the cyrus/bin folder to the boot drive, and chmod’ing the permissions to 777. This also vastly increases the speed of transfer.
GUID not transferred -
Unfortunately this is not only a common problem, but also a painful process to fix. If you have a lot of cases where the user’s GUID number was not brought through it would be worth scripting the following process. Alternatively get in touch if you would like this done for you.
If this has happened for all users it means your server is not properly configured to talk to your LDAP directory.
You need to manually look up the GeneratedUID number for each user where the mail directory (in /var/spool/imap/dovecot/mail by default) is still named with their ‘shortname’ from the LDAP directory and rename their mail directory with the user’s GUID. To look this up from the command line you can use the command:
dscl /Search read Users/USERNAME GeneratedUID
‘chaps’ is a useful command to test your connection with LDAP.
Clients not reconnecting –
I nearly always forget to enable non SSL connections. – I’d start there.
Everything above comes from my experience of migrating Apple mail servers from OS X 10.5 to OS X 10.6, but I’m sure there are others! If you find yourself looking for further help, or remote support, please get in touch, and I would be happy to discus some options.
For further free support I regularly find the guys at afp548.com forum invaluable!
Today Thomson Local (a leading online directory in the UK) has announced its deal with Google to help power Google Maps, by providing data and updates from its directory.
In a media release by Thomson Local’s this morning, Richard Avery, General Manager of New Media for Thomson Local described how this arrangement works well for both their customers and the users of Google Maps.
users want relevant search results with enhanced information, and our customers rely on us to generate quality leads to their business
Thomson Local has been around since 1980 (over 25 years) and focuses on providing directory services to local areas. Since 2003 it’s also been providing services online, rivaling companies like yell.com with over 46 million local searches each month (source), placing particular emphasis on its Pay Per Click, Local Search and Directory Advertising services.
Kelly Allison form Google’s Strategic Partnerships team also praised this agreement saying that it would help with the increasing demand for “local” searches.
Thomson Local’s full media release can be found at:
http://www.thomsondirectories.com/newsarticle.aspx?id=62
For more information on how this might effect you and your business take a look at my previous post: Google Maps and Thomson Local alliance. Should I pay for advertising?
- Oliver
I’ve just had to remind my self where you go to submit a website to the Yahoo directory for crawling – it seems there’s are a lot of sites out there suggesting fantastically complicated methods and broken links. In fact, it’s very simple. The link is:
http://search.yahoo.com/info/submit.html
Once you have submitted your site you should expect a wait of several weeks before your site is crawled. Alternatively you can pay for an express submission (within 48 hours) here.
Simple!
- Oliver
Yet another phone call from Thomson Local. Yet more offers to “increase my web traffic”, “improve my business’ visibility” and “make my key words appear in Google”… Yet another salesman about to leave disappointed. But then he mentioned something interesting. Google Maps.
Google maps is one of the best search engine marketing (SEM) tools for businesses reliant on local catchment areas and clients. And it works both ways – as a user, fed up of trawling through search results pointing to businesses the other end to the country, it quickly and simply shows you local businesses, how to contact them, and how to get to them. For small businesses, in highly competitive markets this can be crucial as getting high on a standard Google search often proves highly time consuming or a hefty expense. You don’t even need a website to benefit from it.
Back on the phone, the sales representative informed me that Thomson Local now provides the data for Google maps. Although it is still possible to add a business directly through Google, details provided by Thomson Local will come up top.
Back in May 2005 Google announced that Thomson Local would be the first European directory publisher authorised to sell advertising through Google AdWords. Ever since, it has been a strong part of Thomson Local’s sales pitch. (Google’s press release). Just before Christmas (2009) Thomson Local data started appearing in the Google maps directory. Indeed if you scroll down to the bottom of a Google Maps search you will see the phrase “Business listings provided by ThomsonLocal.com”.
While the details of this development are yet to fully emerge, at present Thomson Local listings only seem to be getting priority as paid advertisements within Google maps. (Highlighted in yellow as with standard Pay-per-Click adds).
If you don’t currently pay for maps advertisements then you probably don’t need to start. Google’s Local Business Centre still exists and still offers a fantastic range of features and functions, (stats, coupon promotions, video, text and image addition), which used properly can be far more enticing then a plan text link – even if it is a the top. And it’s free!
Whilst the list of results to the left of the maps page is useful, the majority of people still look first to the map itself. It’s your proximity to the client, and the informative nature of your ad, that will make them choose you.
If you are thinking of advertising with Thomson Local have a look here for a full list of their advertising partners.
Clean, straight lines of Poppies, bearing the stark imprint “IN REMEMBRANCE” adorned the bank underneath Malvern College’s chapel on Remembrance Sunday. Raindrops, perhaps aptly, setting on the poppies and crosses create a moving feeling, and detract from the pomp that too often surrounds these Remembrance Day services.
© 2012 Oliver Helm · Proudly powered by WordPress & Green Park 2 by Cordobo.
Valid XHTML 1.0 Transitional | Valid CSS 3
