Virtual mailboxes with courier-imap and postfix

This is the first of an occasional series of posts about Linux systems administration. I’ve been an on-off Linux sysadmin for about, well, my first Linux was Slackware on a stack of 3.5″ floppies. Every now and then I do something “fiddly” and I want to capture these episodes in case I ever need to do it again, or in case someone else wants to and they find this useful.

I run Debian Testing on kenny, the server that hosts http://dannorth.net, http://behaviour-driven.org and a bunch of other websites, blogs and wikis. (Random plug: it’s hosted by the fine folks at SolarVPS. I have no affiliation with them but they rock.) It seems a good balance between Debian Stable – which is rock solid but always about a year out of date – and Unstable, which requires updating far too often and hosed my old server (cartman – can you see a theme here?).

Mostly it Just Works. In particular WordPress and MoinMoin are a joy to configure and use. I get change out of about 10 minutes to add a new blog.

I also use it as a mail server, using postfix for sending and receiving mail and courier-imap for reading it. I tried a number of mail servers and settled on postfix after dismissing sendmail and qmail as just too complicated and exim after staring at the impenetrable documentation one time too many.

I found it pretty easy to set up secure SMTP over TLS and secure IMAP over SSL, but I stumbled at setting up virtual mail addresses. This article is about how to do that using postfix and courier-imap. It’s pretty straightforward once you know where all the moving parts are, but they were less than obvious to me.

Virtual mail addresses

Mostly you send email to bob@address.com and it turns up in bob’s mailbox in bob’s login account. Sometimes you don’t want this. For instance, I host about 20 domain names on kenny, and addresses like info@blah.com or sales@blah.com need to go to specific people. That’s the first type of virtual addressing, known as virtual alias domains.

The second case is “more” virtual – the user doesn’t even need to exist on the server with a regular login account. Postfix puts the mail into a special directory, and courier-imap presents that directory as the mailbox when the user “logs in” over IMAP. This is where all the moving parts come in. This is known as virtual mailbox domains.

Setting up postfix for virtual alias domains

This is the easier of the two, since the mail ends up in a real email address, so there is no corresponding configuration on the courier-imap side. This information came from the Postfix Virtual Domain Hosting Howto.

In /etc/postfix/main.cf add the following lines:

    virtual_alias_domains = dannorth.net
    virtual_alias_maps = hash:/etc/postfix/virtual

This says that postfix will treat the name dannorth.net as a virtual alias domain and will use the file /etc/postfix/virtual to do the mappings. Your /etc/postfix/virtual might look like this:

    # deliver to local account
    dan@dannorth.net dan

    # forward to another mail address
    example@dannorth.net dan@example.com

Once you have this file configured, run the command:

    # postmap /etc/postfix/virtual

to create the hash database that postfix will use, then run:

    # postfix reload

to update the configuration.

Setting up virtual mailbox domains

Ok, there are several moving parts here. We need:

  • a directory to deliver the mail to
  • to tell postfix to deliver it there
  • to enable virtual users in courier
  • to tell courier where the virtual users’ mail lives

In this example, I’ll set up two virtual users, fred and barney, at example.com.

System accounts and directories

The virtual users’ files need to be owned by someone, so we’ll create a “fake” user and group. I’m using vmail for both the user and group names, with uid and gid both set to 5000.

From a root prompt:

    # groupadd -g 5000 vmail
    # useradd -g vmail -u 5000 vmail
    # mkdir -p /home/vhosts/example.com
    # chown vmail:vmail /home/vhosts/example.com

Configuring postfix for virtual mailbox domains

In /etc/postfix/main.cf

    virtual_mailbox_domains = example.com
    virtual_mailbox_base = /home/vhosts
    virtual_mailbox_maps = hash:/etc/postfix/vmailbox
    virtual_minimum_uid = 100
    virtual_uid_maps = static:5000
    virtual_gid_maps = static:5000

This is pretty self-explanatory. It says example.com is a magic virtual mailbox domain, all users and groups map to a fixed number (you can get cleverer than this but I’m not worried about it for now), and that all the interesting stuff is in /etc/postfix/vmailbox. The minimum uid is postfix’s safety measure in case I do something stupid. It means I can’t accidentally let someone have access to system files.

Now let’s look at /etc/postfix/vmailbox:

    fred@example.com example.com/fred/
    barney@example.com example.com/barney/

The magic here is the / at the end of each line. This says to use maildir format (the format courier-imap is expecting) rather than clunky old mbox format. Postfix will create the appropriate directory structure for fred and barney,

Again we create a hash of this for postfix:

    # postmap /etc/postfix/vmailbox
    # postfix reload

Phew! That’s the postfix side done. Now stop for a cup of tea.

Configuring courier-imap for virtual mailboxes

On Debian, courier imap runs as three executables, each with separate init.d scripts. courier-imap and courier-imap-ssl are the imap servers themselves (I run courier-imap bound to localhost for webmail). courier-authdaemon is the chap that does all the authentication. That’s the one we’re interested in.

Firstly, we need to enable virtual users. In the file /etc/courier/authdaemonrc you need to make sure your authmodulelist setting contains authuserdb as one of its authentication mechanisms. Mine looks like this:

    authmodulelist="authuserdb authpam"

Don’t forget to tell the auth daemon you’ve made a change:

    # invoke-rc.d courier-authdaemon reload

Courier uses a file called /etc/courier/userdb to store virtual user mappings, but you don’t usually edit this file yourself. It has an arcane format (using tabs and pipes as delimiters) and should be left well alone. Instead, courier provides you with some command line tools to manipulate it.

To create an entry for fred, we do this:

    # userdb fred set uid=vmail gid=vmail home=/home/vhosts/example.com/fred mail=/home/vhosts/example.com/fred

(That should all be on one line – your browser might wrap it.) Then set a password for fred:

    # userdbpw -md5 | userdb fred set systempw

Do the same for barney. Finally we build the hashed user database that courier will actually use:

    # makeuserdb

Note: don’t forget to run makeuserdb after making any changes to the virtual user data otherwise courier won’t know.

Testing the configuration

Firstly, try sending an email to the virtual user. Postfix should create the maildir structure under example.com/fred. Then try connecting to courier to read the mail. If you find you are getting authentication problems from the courier side, you can try setting DEBUG_LOGIN=1 in /etc/courier/authdaemonrc and restarting the auth daemon. Don’t forget to switch it off again once it’s working.

30 comments

  1. Concerns about NBehave…

    This is an attempt to answer Scott Bellware’s concerns and questions about NBehave . Scott, first off…

  2. [...] Virtual mailboxes with courier-imap and postfix This information came from the Postfix Virtual Domain Hosting Howto. In /etc/postfix/main.cf add the following lines:. virtual_alias_domains = dannorth.net virtual_alias_maps = hash:/etc/postfix/virtual … [...]

  3. Dude, you should try dovecot in place of Courier IMAP. Not only is it lighter and generally spiffier than Courier, but postfix understands how to talk to dovecot’s auth process; that means you can set up a user DB for dovecot, then easily enable authenticated SMTP for your users by telling postfix to talk to dovecot.

    Missing ya; hope you’re well.

    1. Missing you too :)

      I’ll take a look at dovecot when I get a chance. It’s going to be a while though..!

  4. Thank you very much for this, i have been looking for a nice How-to like this.
    One problem though: in the /home/vhosts/example.com/fred , i had to make the maildir by myself, including the subdir “cur” “new” “tmp”
    For the first user it automagically created by itself, but for the other users i had to do it myself.

    Regards,

    G

  5. Hey, Great tutorial thanks. Just one question, does it work for courier-pop too? Any additional settings required? I am under the assumption that courier-pop and courier-imap use the same internals?

    1. Hi Matthew.

      Courier POP3 (and POP3/S) will operate in exactly the same way. I prefer IMAP because then the server always holds your email and your local mail client is just a copy. So you can use a webmail client (I use squirrelmail) if you can’t get to your usual computer, and you’ll still have access to all your mail.

      In any event, I took Steve Purcell’s advice (a few comments above) and switched to dovecot. He’s a very smart man and dovecot is lovely. Its authentication module integrates really nicely with postfix which makes managing virtual user accounts a dream.

  6. Made me smile to find your blog when searching for the answer to a postfix problem. It was #4 when I searched for ‘postfix virtual user database’. Anyway, hope things are going well. Funnily enough, this post answered my question ;-)

  7. Hey Dan,

    Just a quick note – took your/Steve Purcell’s advice and switched to Dovecot – got it all working great in the end with postfix running off the same authentication system.

    Wrote a couple of bash scripts and its really easy to manage too :)

    Thanks,
    Matthew

  8. [...] DanNorth.net » Virtual mailboxes with courier-imap and postfix [...]

  9. Thanks much for the very helpful write-up.

  10. Hi Dan,
    have virtual and unix account mailboxs running at the same time, i mean once virtual is enabled does it mean all mail boxes must be virtual?
    thanks,
    jon

    1. Hi Jon.

      Since I wrote this post I have moved to dovecot for IMAP. It plays nice with postfix (which has built in support for dovecot’s authentication module). This means that dovecot’s auth configuration works for both sending via postfix and reading with IMAP or POP3.

      With dovecot you can mix real and virtual accounts by using different combinations of userdb and passdb. Dovecot will use the first match it finds. For instance I have these two stanzas for users:

        userdb passwd-file {
          args = /etc/dovecot/vhosts
        }
        userdb static {
          args = uid=vmail gid=vmail home=/path/to/mail/%u mail=/path/to/mail/%u/Maildir
        }
      

      The file @/etc/dovecot/vhosts@ contains entries for virtual users similar to @/etc/passwd@, but with extra parameters at the end (for example userdb_mail=maildir:~/Maildir).

      If the user doesn’t appear @/etc/dovecot/vhosts@ they are assumed to be a regular account with their email stored under @/path/to/mail/USERNAME/Maildir@.

      There’s probably a follow-up blog post in all this but hopefully this should be enough for you to go on.

      Cheers,
      Dan

  11. Here:

    > Don’t forget to tell the auth daemon you’ve made a change:
    > invoke-rc.d courier-authdaemon reload

    I get:
    Usage: /etc/init.d/courier-authdaemon {start|stop|restart|force-reload}
    invoke-rc.d: initscript courier-authdaemon, action “reload” failed.

    So maybe you meant “force-reload” or “restart” instead of “reload”?

  12. Also this fails:

    > userdbpw -md5 | userdb fred set systempw

    I get:
    Password: Can’t open /etc/courier/.lock.userdb: Permission denied at /usr/sbin/userdb line 83.

    even though I do it with sudo

  13. Ok I figured out the latter.
    The problem was I was doing:

    sudo userdbpw -md5 | userdb fred set systempw

    But this way the pipe is “outside of the sudo”. So I got it to work by doing this:

    sudo su
    userdbpw -md5 | userdb fred set systempw

  14. I’ve followed all the steps. The only difference is I have the mail dirs in /var/spool/virtual/example.com/user1 (user2, etc) because I had followed another tutorial for setting up postfix.

    However everything else is the same. Postfix works fine and actually delivers mail as expected (I have checked the files). However I can’t get courier to work. When I log into the imap server I get this error:

    * BYE [ALERT] Fatal error: Account’s mailbox directory is not owned by the correct uid or gid:
    Connection closed by foreign host.

    The mail directory do belong to the user vmail of group vmail which are the uid y gid that are set in the userdb file….

    Any idea?

    thanks
    m.

    1. OH SHIT! Here is the error:

      # userdb fred set uid=vmail gid=vmail home=/home/vhosts/example.com/fred mail=/home/vhosts/example.com/fred

      You have to use the NUMERIC uid and gid, not the user and group’s name. It seems funny to me that nobody has mentioned this: does everybody understand as obvious that when you write “uid=vmail” you mean have to replace “vmail” with the numeric uid of user vmail?

  15. Great post!! Comments by matteo helped too. Everything seems to work well. Just one comment to newer users like myself: do not use virtual alias domains and virtual mailbox domains configurations together in main.cf – it is either or!!

    My mails are being delivered to virtual mailboxes however I am perplexed as to how to access them. I tried using squirrelmail but I can only access mails for my hosted domain and not for my virtual mailboxes. Any suggestions??

    1. It does work with both virtual aliases and virtual mailboxes, but you can’t mix them within the same domain. A domain must be either a virtual alias domain or a virtual mailbox domain.

      Soon after I wrote this I took Steve Purcell’s advice and shifted to Dovecot both as an IMAP server and for Postfix authentication. It made things a lot simpler and I was able to use the same password file for both Postfix (SMTP) and Dovecot (IMAP) authentication.

  16. I changed my mta from courier to Dovecot. I am not sure how I would implement userdb, userdbpw and makeuserdb in Dovecot. I tried using command line but it does not seem to work. Any help? Thanks

    1. I seem to remember using the Dovecot wiki to set it all up, and I don’t think it was too painful. Turn up all the logging and check the log files (also, running dovecot in the foreground rather than as a daemon might give you some clues).

      For the last couple of years I’ve had all my mail hosting on Google hosted domains, so it’s been a while since I’ve looked at this.

  17. I finally got my virtual mailboxes running. I can access my mails both through pop3 and IMAP. I am using squirrelmail to get my IMAP mails. I found a very detailed and very easy to implement article for virtual mailboxes using dovecot: https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto, this includes shell scripts for adding and deleting users . Just one pointer, make sure you configure squirrelmail to reflect that you are using Dovecot in pre-defined settings for specific IMAP servers.

  18. OK, it works well.
    But if I want to install some kind of webmail, I need SMTP login on the server.
    How can I make it with virtual postfix users?

    1. Squirrelmail works quite well, login into your account using your email instead of just user name. I don’t think you need special configuration for squirrelmail except indicate which MTA (such as dovecot) you are using in squirrel configuration in pre-defined settings.

  19. This is an old blog post now but still useful. It helped me get setup with Postfix and Courier userdb + SASL for SMTP AUTH (with a little teeth pulling as can be seen on my Server Fault question: http://serverfault.com/questions/405808/postfix-smtp-auth-not-working-with-virtual-mailboxes-sasl-courier-userdb/405812).

    1. Hi Greg,

      I’m glad you found it useful. Nowadays I would use dovecot and postfix if I were going to host my own mail server. Dovecot has a great authentication module that integrates neatly with Postfix and handles virtual mailboxes and entire virtual domains. Although my first choice is to outsource mail hosting to Google Apps.

  20. When I log into the imap server I get this error:

    BYE [ALERT] Fatal error: No such file or directory: No such file or directory
    Connection closed by foreign host.

    1. Ok it’s work :)

Follow

Get every new post delivered to your Inbox.

Join 458 other followers

%d bloggers like this: