I promised an Amazon EC2 AMI with imapsync and a few Gmail migration scripts. See the Gmail to Google Apps Email Migration post for background information. I’ll jump right in with a procedure to get you started with your own migration:

  1. Run the instance:  ami-01789d68
    ec2-run-instances ami-01789d68 -k your-own-keypair
  2. Use your key to authenticate through SSH using the username “gimapsync”. See Running an Instance in the Amazon EC2 Getting Started Guide for more information.
  3. Edit the ‘run-imapsync-justfoldersizes’ script:
    nano -w run-imapsync-justfoldersizes
    • Change alias@gmail.com to your ‘from’ email address
    • Change alias@google-apps-domain.com to your ‘to’ email address
  4. Edit the ‘run-imapsync-all’ script the same way
  5. Edit the ‘run-imapsync-folder’ script the same way
  6. Create passfile1, which should contain the password for your ‘from’ email account
    nano -w passfile1
  7. Create passfile2, which should contain the password for your ‘to’ email account
    nano -w passfile2
  8. Test it out with
    ./run-imapsync-justfoldersizes
    • It should connect to both accounts and list each imap folder and its size
  9. If you are feeling lucky, just run
    ./run-imapsync-all
  10. Otherwise, go one folder at a time with commands like:
    ./run-imapsync-folder "[Gmail]/Sent Mail"

Feel free to edit the run-imapsync* scripts to fit your needs. There are many command-line features of imapsync that I have not made use of here. You may be interested in the ‘--maxage N’ parameter, for example. It only transfers email that are newer than N days old.

Let me know how it goes!

Comments

clark

Wow. Thanks for writing this up and setting up the EC2 image. This was a life saver. Still running the conversion, but it seems to be going off without a hitch!
"Cowboy" Ben Alman

You need to be sure that when editing the command-lines, you not only change "alias@gmail.com" to your old email and "alias@google-apps-domain.com" to your new email, but that you also change "alias\@gmail\.com" and "alias\@google-apps-domain\.com" as well. It might be a good idea, in the future, to handle the escaped and non-escaped versions programmatically, and store the email address in shell variables (to make editing them easier) Awesome job though, thanks!! - Cowboy
Scott Mottonen

Thanks for setting up this image. Other than the problem of imapsync dying periodically during a long transfer, it works very well! Getting imapsync to run on my own Mac wasn't going well. The ec2 image saved me a lot of time.
ff

Hi, would you mind sharing the scripts and your version of imapsync? I have the problem that if I sync my Gmail account with my GApps account some of the attachments get broken. Somehow there seem to be some missing bytes at the end of an email... Thanks in advance
Sam

ff, the script is here - http://www.thamtech.com/blog/2008/03/29/gmail-to-google-apps-email-migration I'm having a similar problem syncing my Gmail with a local IMAP server, or my ISP's IMAP server. The end of messages are truncated.
George Davis

Great tool - thanks for sharing! Have you done any other interesting AMI work recently? G
Bruce

Thanks for the tool, but when I try to log in using Putty, I get a message that there is no Authentication Method. Is there a way to set it up (without a key pair), but using a default username and password? please?
Tyler

Bruce, I'm sorry to hear you are having trouble using a key pair. Setting up the AMI and publishing a default username and password is a bad security practice and could result in someone finding, publishing, and/or using YOUR gmail passwords. Take a look at the "Running an Instance" section (http://docs.amazonwebservices.com/AWSEC2/2008-02-01/GettingStartedGuide/index.html?running-an-instance.html) of Amazon's Getting Started Guide. It explains how to set up a key pair and has a note about making it work with Putty. Tyler
Bruce

Tyler, thanks for the reply. I'm using Elasticfox (a Firefox Extension for Amazon EC2, no need to use command line tools, or java), so in 1 click I run the instance, but I can't SSH. What I meant with defalt username/password was to access the instance. Only I know the DNS address of my instance, and it's only for a few minutes/hours. I would login with a default root/pass, run the scripts, the delete the instance. By the way, I have about 800 MB in Gmail.
Ami

FWIW, http://google.com/support/a/bin/answer.py?answer=61369 seems to be working fine for me migrating from an @gmail.com account to a GAFYD account with no external programs/processes (all google-hosted).
Bruce

I ended up following these steps http://glomerate.wordpress.com/2008/08/20/migrating-from-gmail-to-google-apps/
Waldo Jaquith

Well, that was very useful. (And my first foray into AWS.) Thank you for providing this.
Larry McJunkin

Any ideas for moving back from Google Apps Gmail to Regular Gmail and keeping all these things intact? I'm tired of waiting for Google Apps to include Google Labs and Themes and they're more important to me than Google Apps. Thanks. Larry
Tyler

Larry, I think the same procedure should in reverse without any changes. Just use your Google apps login as the source and your @gmail.com login as the destination.
Hamden

Awesome job!!
Oscar

Thank you for sharing this, the main thing keeping me from switching my email to my domain name was leaving all the old emails behind, and this has solved my problem big time!
Stuart

Thanks for this, I had similar requirements for email import and this was almost spot on. I imported label by label, which meant that about 400 out of 7500 emails weren't imported.. however I think it's a reasonable trade off given how little I go back through all that old mail. :)
Dave C

Tyler, Thanks! I also had to run the script several times in order for it to migrate ALL my emails. And also had trouble getting PuTTY to work properly, until I realized I had to log in with the "GIMAPSYNC" login. Once I changed the login (and key pair, just to be consistant), it all worked fine. Thanks again, Dave