23 Jan

Email archive, Google style!

I managed to get a little email archiving system working which I am finding to be potentially quite useful in the future. The beauty of it is that every email going in and out the system is saved onto a google gmail account, which harnesses their power of filtering, searching and labelling!

This setup was specifically aimed at a PLESK-run box but I’m sure it would work on any box with a qmail/qmail-scanner enabled mailserver, actually any MTA with a Maildir format.

Firstly, you must install qmail-scanner onto your plesk system. This is quite straightforward and I have a tutorial here to help you install qmail-scanner onto a plesk server.

Once you’ve done that, go to /etc/qmail-scanner.ini and edit the line that says

ARCHIVE=”no”

to

ARCHIVE=”yes”

and then reconfigure qmail-scanner by running

qmail-scanner-reconfigure

Now you’ll start receiving archived email into folder path /var/spool/qscan/archives/new/

The trick to move these to gmail is set up a cronjob to move these archives to an email account you create on your server via PLESK/qmail and then tell gmail to retreive it via pop.

So, firstly you must create an email account. Doesn’t matter what it’s called, where it is or anythinng else, just create an email account as you would on a PLESK domain of your choice with mailbox enabled and a decent password.

Once you do this, create a script somewhere in your system with these contents (I chose /var/qmail/ for this example)

#!/bin/sh
mv /var/spool/qscan/archives/new/* /var/qmail/mailnames/<yourdomain>/<mailname>/Maildir/new/ > /dev/null 2>&1
chown -R popuser:popuser /var/qmail/mailnames/<yourdomain>/<mailname>/Maildir/new/

Next create a cronjob on your server (Server > Scheduled Tasks) and run that. I run it under root so that it doesn’t mess up on setting permissions. Depending on how much mail you get you need to run this quite regularly, otherwise mv just won’t work! In my case, once a day wasn’t enough so I just set it to every 5 minutes… probably less intensive to do in increments anyway.

So now you could download these to any pop or imap account, but we choose gmail because of the storage space, reliability, and speed at sifting through huge archives of mail. You probably want a new gmail account for this I imagine, and in many cases you may want to splurge on a paid plan for more space, depending on how much email you get. I’ll leave that to you.

In your desired gmail account go to Settings > Accounts and you’ll see an option (2nd one down at time of writing) called “Get mail from other accounts:“. Click on the link to add a new account.

Qmail Archiving to Gmail - Step 1

The first windows asks for the email account you just set up on PLESK. The second window will look like the following. Change the values in the textbox to match your server and email account.

Qmail Archiving to Gmail - Step 2

That’s it! Sit back and watch the emails roll in, and make the best of gmail’s power! You can create a filter for every email account you have on the server and apply a label automatically, so if you ever need to access a lost email (or snoop on your employees) you won’t have to wait for ages as your poor email client tries to go through thousands of emails to get what you want.

If you liked this post, why not share it?

  • Print
  • Digg
  • del.icio.us
  • Facebook
  • Mixx
  • Google Bookmarks
  • Reddit
  • RSS
  • StumbleUpon
  • Technorati
  • Twitter

Related Posts

  • Craig

    Do you know if this or your previous post will wotk on a multi-domain Plesk server? We have some domains that need archiving and some that dont. I dont want everything sent off or archived, just specific domains.

  • http://netweblogic.com NetWebLogic

    The best way to go about it would be to edit the /etc/qmail-scanner.ini file and use a regular expression in place of ARCHIVE=”yes”. Check out http://qmail-scanner.sourceforge.net/configure-options.php for more information on that.

    In your case, I suppose you could write something like:

    ARCHIVE=”.*@(domain1\.com|domain2\.com)”

    Don’t forget to run qmail-scanner-reconfigure after editing!

  • http://netweblogic.com NetWebLogic

    Edit: I changed the crontab to run as root, as there were problems with permission setting. Post also modified to reflect this.

  • Chris

    Hi there! First of all — This solution is absolutely AWESOME!!! For anyone who is interested you can utilize this solution to actually by as much as 1 TERABYTE of storage for ONLY $259 per YEAR!!!

    See this page for more info: https://www.google.com/accounts/PurchaseStorage

    NetWebLogic (or anyone who knows a solution): I could REALLY use a bit of help here so I appreciate anyone providing me with some insight or a solution.

    Here is a brief background… I have a client of mine in the financial service industry who requires full email archiving to meet compliance standards.

    OK… so now to my issue.

    What I have found out is that federal regulations mandate that all these sent/received emails are NOT to be altered in any way shape or form. After reviewing this in detail I found out that what I actually need to do is include the full original email message as an attachment.

    So… What I an I am assuming anyone is my situation needs is a way to modify this script in some way by which each and every email that is archived actually includes the COMPLETE ORIGINAL sent or received email as an attachment.

    In an ideal situation I would actually like to have each archived email include the complete original email as an attachment, ensure this archived email shows the original “from” email address and “to” (as well as cc and bccs) and the original subject line prefaced by “ARCHIVE –”… Aside from actually attaching the original email message I think the email message body should show summary information of that email with the qmail scanner summary information on top, along with the first maybe 500 characters of the message body and thereafter the full email header.

    Does anyone here have the ability to help me modify this system to accomplish this. I would assume this would actually be fairly simple to modify but I unfortunately just don't have the experience needed to modify this script to efficiently create a new email message, attach the original one and include specific info from the original message.

    Thank you guys so much in advance for your help!

    Chris

  • http://netweblogic.com Marcus

    Not sure how simple this is… but not really willing to find out! You'll probably have to change the qmail-scanner source code to acheive this.

    I'd say you have two options: 1. find a python programmer and have it altered. 2. find another solution, there must be ready-made products for the accounting industry.

    Best,
    Marcus