20 Sep 2017

Dovecot Push Notifications for iOS

Native Apple push notifications for your self-hosted mail server.

EDIT: Apple has recently announced that they are dropping mail suport from macOS Server Fall 2018. Once your APNS certificate expires after one year, the push notifications will stop working, and it seems like we won't be able to get new certificates from the Server app. Due to this news, I've stopped maintaining the patchset, since it's a fair amount of manual work to update it for each Dovecot release. Feel free to read on if you're interested in how this cool trick works though!

Using a patched version of Dovecot and a small Perl script, it's possible to get native Apple push notifications for new mail using the stock iOS mail client. This is accomplished by mimicking Apple's XAPPLEPUSHSERVICE IMAP extension for Dovecot used in macOS Server, whose source code is published on the Apple Open Source website.

If you're running Dovecot on FreeBSD, this guide will be extremely easy: you can just use my patched Dovecot port here. It should work just as well on other Unix-like operating systems with a bit of manual work.

Acknowledgements

All the credit for this goes to Matthew Powell on GitHub. He wrote the Dovecot patch and notification daemon to get push notifications working. I just packaged everything into nice FreeBSD ports.

Much of this knowledge was discovered a few years ago by Stefan Arentz in his dovecot-xaps-plugin project.

What You'll Need

Installation

The setup has two components: (1) a patched Dovecot installation to handle the APNS token negotiation over IMAP, and (2) a small Perl daemon which sends notifications to Apple's push servers whenever Dovecot receives a new message in a user's inbox.

On FreeBSD, you can install both of these components by building my mail/dovecot port with the APNS option. This should automatically pull in my mail/dovecot-apns-daemon port as a dependency:

# clone my custom ports repo
git clone --- /usr/local/custom-ports

# symlink my custom ports into your ports tree
# (I need to find a cleaner way to do this)
rm -rf /usr/ports/mail/dovecot
ln -s /usr/local/custom-ports/devel/p5-Privileges-Drop   /usr/ports/devel/p5-Privileges-Drop
ln -s /usr/local/custom-ports/net/p5-Net-APNS-Persistent /usr/ports/net/p5-Net-APNS-Persistent
ln -s /usr/local/custom-ports/mail/dovecot-apns-daemon   /usr/ports/mail/dovecot-apns-daemon
ln -s /usr/local/custom-ports/mail/dovecot               /usr/ports/mail/dovecot

# build the patched dovecot
cd /usr/ports/mail/dovecot
make config  # ensure the APNS option is selected
make reinstall clean

The pkg-message tells you how to export the magic APNS certificates from your macOS Keychain to your FreeBSD server:

Apple Push Notifications require a valid certificate keypair from
a working copy of OS X Server. Export the following APNS certificate
from your OS X keychain:

APSP:com.apple.servermgrd.apns.mail 

By default, the dovecot_apns daemon will look in the following
locations for the certificate key pair:

/usr/local/etc/dovecot-apns/mail.crt
/usr/local/etc/dovecot-apns/mail.key

If you exported a .p12 file from your OS X keychain, you can
convert it to the proper PEM format with the following commands:

openssl pkcs12 -in push.p12 -clcerts -nokeys | sed '/BEGIN CERTIFICATE/,$!d' > mail.crt
openssl pkcs12 -in push.p12 -nodes -nocerts | sed '/BEGIN RSA PRIVATE KEY/,$!d' > mail.key

The dovecot_apns daemon must be running for device registration and
push notifications to work. Enable it in /etc/rc.conf:

echo 'dovecot_apns_enable="YES"' >> /etc/rc.conf

As well as how to configure Dovecot:

Set the following variable somewhere in your Dovecot configuration:

aps_topic = com.apple.mail.XServer.XXXXXXXXX

replacing the X's to match the UID field in your APNS certificate.

If you exported your mail push certificate as push.p12, you can get your aps_topic value with the following command:

openssl pkcs12 -in push.p12 -clcerts -nokeys -nomacver | grep -o 'com\.apple\.mail\.XServer\..*' | awk -F/ '{print $1}'

I have mine configured like this:

/usr/local/etc/dovecot/conf.d/90-apns.conf
aps_topic = com.apple.mail.XServer.xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

Once you have your patched Dovecot installed, your APNS certificates in place, and your Dovecot configuration updated, give the daemons a kick:

service dovecot restart
service dovecot_apns start

Dovecot and the APNS daemon will both log to /var/log/maillog. Toggle Airplane Mode on your iPhone to force the IMAP connection to reset. You should see the token negotiation logged in your mail log. On your iPhone, your mail account's fetch schedule should automatically change to push in Settings → Accounts & Passwords → Fetch New Data.

Enjoy the battery savings and instant email notifications! I'll do my best to keep my Dovecot port on GitHub in sync with whatever version is in the official FreeBSD ports tree, though updates may not be instantaneous.

If You're Not Using FreeBSD

If you're using Linux or something other than FreeBSD, you should still be able to get this working. You'll just need to apply the Dovecot patch (currently valid for Dovecot 2.2.32) before building Dovecot from source. You'll also need to install the Perl dependencies for the pushnotify.pl script using CPAN or your distribution's package manager. Finally, patch pushnotify.pl using my patch and run it as a systemd unit file or tmux session or something.

Or, just follow the instructions on the original repo, it's probably easier. 😃

But How Does It Work?

When your iPhone connects to an IMAP server, it checks for XAPPLEPUSHSERVICE in the server's CAPABILITY list. If this special string is found, the iOS Mail app assumes that it has connected to a macOS Server, which should support a custom song-and-dance of IMAP commands to register for push notifications.

In our case, we have simply patched Dovecot in a similar fashion to Apple's macOS Server. When your iPhone sees the XAPPLEPUSHSERVICE capability, it sends a special command to register its device token for push notifications. Our patched Dovecot intercepts this token and hands it off to our Perl APNS daemon over a local UNIX socket (/var/run/dovecot/apns). The APNS daemon stores a map of email accounts to APNS tokens in a flat file database, which you can view at /var/db/dovecot-apns/devices.

When an email account receives a new message in its inbox, our patched Dovecot sends another message over the APNS daemon's UNIX socket. The daemon looks up the email account's associated APNS token. It then uses the mail.XServer keypair you exported from your macOS keychain to sign a notification message, which it sends to Apple's push notification server. Your phone then receives a push notification regarding the new email over a persistent, highly optimized connection to iCloud.

As far as I can tell, buying macOS Server is the only way to get the necessary mail.XServer certificate to send push notifications to the stock iOS Mail app. Other companies that advertise native iOS push email (such as Fastmail) probably have a working relationship with Apple, which allows them to get more permanent APNS certificates—the ones provisioned to macOS Server expire after one year and must be re-exported.

Conclusion

I can't guarantee how robust this setup is for "production" use, but it works great for my family's email server. I do usually have to restart the APNS daemon whenever I restart Dovecot. Also, remember your APNS certificate will need to be renewed after one year.