diary at Telent Netowrks

MuDDLe, a faster simpler maildir downloader#

Sun, 02 Feb 2014 23:27:24 +0000

One of the services running on my old Bytemark VM was the Dovecot IMAP server. When I started thinking about configuring it on the replacement box I realised that I don't actually need IMAP these days, so, er, why bother? So I didn't. But what I do need is a way of getting the Maildir on that machine onto other machines, and everything I looked at to do this job (other than rsync) was fearsomely complicated because it also catered for a zillion other file formats and/or transports. Or because it wanted to sync in both directions, which I don't really care about that much.

So, muddle

In essence, what it does is this: connect to the remote host, find list of files in cur/ and new/, compare with similar list on local host, create tar stream of differences, transfer it, unpack each transferred file into tmp/ and atomically rename into cur/ when done.

This adds up to one transfer of file names and a second transfer of all the file contents. Each of these is one-way and distinctly non-chatty, so it should have reasonably good network performance characteristics and you can use ssh compression if your network bĂȘte noire is bandwidth itself and not just latency.

It might not be as simple as possible, but at the same time it might also be simpler. For example, and as alluded to above, the download is one-way only, so it won't e.g. update the server to mark messages as read. If you care about that stuff, this is not for you.

(Why not rsync? It can't detect that a rename from new/foo to cur/foo:2, is a rename, so treats the latter as a new file. Which is a teensy bit suboptimal)