Blog » HOWTO: CiviMail Return Channel for Multi-site Servers

Setting up the CiviMail return channel is a notorious pain in the ass. The return channel is the mechanism by which email is passed back into the CiviMail system. It's necessary to allow replies and bounces to be handled by CiviCRM, making it fairly essential to the successful use of CiviMail.

The official way of setting things up is unfortunate in a couple of respects: it requires the installation of a customized version of the amavisd-new content-filtering daemon*, and it cannot handle multi-site installs. The "alternative" return channel implementation, although rumoured to not scale as well, requires no additional software, and can be easily adapted for servers hosting multiple domains that use CiviMail.

So to handle multiple CiviMail domains on one server, I use the alternative imap2soap.pl script. This approach requires using catch-all mailboxes for the domains in question. These email addresses will receive the VERP-formatted bounces and replies; a cron job regularly polls them, running a script that allows the mail to be passed through to CiviCRM for processing.

I have implemented this approach a few times now under various circumstances, and provide instructions below for both cPanel and command-line users.

Instructions

  1. Create a subdomain for the exclusive use of CiviMail. This is optional, but recommended, because the catch-all address will receive all the mail not otherwise delivered to that domain. If you create a catch-all for your main domain name, spam and misaddressed mail will land in your civimail inbox unnecessarily. (The mail-processing script ignores non-CiviMail messages, but they still cost you processing time and storage space.)
  2. Create IMAP accounts for your catch-all boxes. These could be Gmail accounts or whatnot. I handle this internally on our IMAP server.
  3. Create a Drupal user on each site with permission to access CiviCRM.
  4. On your mail server: set the new accounts as catch-all for the various domains.

    In cPanel, this can be done from Mail > Default address.

    If you use postfix, edit /etc/postfix/main.cf and add:

    virtual_mailbox_domains = civimail.domain1.ext, domain2.ext, ...
virtual_alias_maps = hash:/etc/postfix/virtual

    and create /etc/postfix/virtual along these lines:

    @civimail.domain1.ext	civimail_domain1@mail.example.com
@domain2.ext		civimail_domain2@mail.example.com
...
  5. On your web server: download the CiviMail alternative return channel scripts and sample config files. Create a config file for each domain, entering the correct info for the relevant email account and Drupal account. To keep things simple, I put the imap2soap scripts in /usr/local/CiviMail/bin/ and the config files in /usr/local/CiviMail/etc/.

    For the purposes of the script that follows, name the script after the domain, ie:
    /usr/local/civimail/etc/civimail.openconcept.ca.conf

    Note that the imap2soap script requires the following perl modules to function:/p>

    • SOAP::Lite
    • Mail::IMAPClient
    • Mail::Verp
    • HTTP::Cookies
    • MIME::Entity
  6. On your web server: Set up a cron job that regularly calls the CiviMail processing script for each domain. My cron entry looks like this:
    11,41 *  *  *  *  /usr/local/CiviMail/bin/civimail_cron.sh

    And the civimail_cron.sh script looks like this:

    #!/bin/bash
DOMAINS="civimail.domain1.ext domain2.ext"
for DOM in $DOMAINS; do
  /usr/local/CiviMail/bin/imap2soap.pl -q -w 1 -l 4 -L 1 /usr/local/CiviMail/etc/$DOM.conf
done

    Note that the arguments passed to imap2soap.pl are documented in the script itself:

    # Command line options
# -q       -- quiet; do not output anything if there are no errors
# -n N     -- submit at most N messages and quit
# -w N     -- wait N sec before processing each message
# -l N     -- quit if load average >= N
# -L N     -- ignore -w if load average < N

    (cPanel users can create the script somewhere not in a web-facing directory and then set up the cron job using the "Cron Jobs" item under "Advanced".)

  7. Tell CiviCRM to use this domain for outgoing CiviMail: under Administer CiviCRM > CiviMail > Domain Information set the "Email Domain" to the domain for which you set up the catch-all

When all is said and done, replies and bounces for each domain will be reported back to CiviCRM.

In order to actually see the replies, you will have to check the catch-all email account with an IMAP client. Note that messages in the inbox have not yet been processed by CiviMail; look to the "Processed mail" subfolder for items that have already been recorded by the system and that may need replies, etc.

* Amavis is a great tool. My annoyance comes from how CiviCRM chose to use it. A hacked version of the main amavis daemon script must be installed, forcing sysadmins to use an arbitrary and out-of-date version (CiviMail requires version 2.4.2, which is over two years old at the moment). Moreover, amavis is highly modular: why didn't anyone write the CiviMail functionality as an amavis plugin instead?

Section: 

Topic: 

Code: