El Reg drills into Office365: Mass email migration
Hands-on with Trevor
Spinning up a new instance of Office 365 to provide email for a brand new domain is easy; migrating email from other domains is not.
If you are a systems administrator who can count to 10 more or less accurately you are probably ready to dive in but the everyday user may find it a bit of a struggle.
Let's take a look at the three basic categories of email migration: PST import, IMAP import and Exchange migration.
Before you do any migration into Office 365 you need to have the DNS in a half-configured state. You need to add the domain name to Office 365 and verify ownership, but you do not want to actually transfer over the DNS records to point to Office 365 until your user accounts are ready for the migration.
Once you have picked your migration type, made backups, scheduled the migration time, run a test, made some backups, alerted your users, made some more backups and then backed up your backups offsite, you are probably good to start.
In the case of PST and IMAP imports, you create the users and then import their data. In these scenarios you run user creation, repoint the DNS (so all new mail is delivered to the Office 365 servers) and then import the old data.
In the case of Exchange, you would perform the migration first, leave it in sync mode and then repoint your DNS at Office 365. The sync widget will keep on synchronising until you are ready to finish up.
A PST import is fairly simple. If your employees are using Outlook then their emails can be saved as PST files. This contains all the email, calendar items, contacts and whatnot that live in their mail client. The files also contain the structure of folders, sub-folders and so forth.
The simplest way to perform a PST import is to spin up a new Office 365 instance, add your domain, change your DNS, set up your users and then pull the PST files filled with email from your old mail setup one at a time.
Connect up your Outlook client to the new Office 365 mail service and import your mail. It takes a while but all your information will be pushed into the Office cloud one item at a time. If you have only a handful of users this is the quickest way to migrate your mail.
As you can probably tell, however, there is no way this scales. Yet for some arrangements – notably deployments reliant on POP-based email – PST imports are the only way to go.
Microsoft is aware of exactly how laborious this can be so it has created Microsoft Exchange PST Capture.
Boiling this one down, you proceed more or less as above: cut the DNS over to the Office 365 install, have everyone connect up to their old email provider to download a final round of email into Outlook and then get everyone to close Outlook down.
It should be noted that you can bulk-create users in Office 365, via CSV, so you don't have to sit around hand-creating user accounts to import into.
Next, you install a PST capture agent on each site (managing them all through the PST capture console) and this sweeps your network for PST files.
The PST capture central service keeps a list of all the PST files you have found and holds your hand through the whole process of merging those PST files into the accounts you created for your users.
IMAP import is messy and finicky. When it works, it works brilliantly, but reality doesn't always do what it is supposed to.
There are many different IMAP servers and they don't all speak quite the same language. My quixotic struggle to get Outlook 2003 and Gmail to cooperate in a friendly fashion serves me as a daily reminder of that.
It is for this reason that Microsoft has an “officially supported” list of IMAP servers it will import from: Courier-IMAP, Cyrus, Dovecot, UW-IMAP and Microsoft Exchange (2000 and newer.)
That is not to say that Office 365 won't import from other IMAP servers – it generally will – but these are the ones that have been extensively tested.
With untested IMAP servers you might run across some bizarre bug if your message has a strange character in the subject or other edge cases the server is not good at handling.
There are also plenty of issues if you are importing from IMAP servers that impose disconnection throttles on sessions with more than a certain number of transactions per minute. It short, IMAP importing from untested servers is not for the fainthearted.
Once you have the server issues under wraps, the import process is not that bad. If you have an administrative user who can access all the mailboxes on your IMAP server, great; if not, you will need to run around and get credentials for each user.
You then put all the credentials into a CSV, for which Microsoft has done a fantastic job of documenting the format and specifications.
You create an “IMAP migration endpoint”, which is a fancy way of saying “script that shuffles messages from A to B”. This will iterate through the CSV, connect up to the IMAP mailboxes and copy mail from the IMAP server into Office 365.
Every 24 hours until you stop the migration it will re-check the IMAP server to see if any new items have been delivered there. Once you are done you simply point the DNS at Office 365 and delete the migration batch.
This is more complicated than the other two methods but still far preferable. If you have your how-to guide by your side, it is ultimately much less hassle. I have done just about every mail migration possible into Office 365 and the Exchange migrations end up with the fewest hiccups and the least expenditure of time.
The internet is full of walk-throughs. The best ones I have seen are from Office365Support.ca. It has top-notch step-by-step guides for Exchange 2003, 2007, and nearly everything else about Office 365 you ever wanted to know. It is an excellent resource for any prospective Office 365 admin and should be bookmarked as an example of technical communications excellence.
The problem with internet walk-throughs is that you still need to know which questions to ask to find the guides you need. Exchange migration to Office 365 comes in a few different flavours.
This is because Office 365 and Exchange are not mutually exclusive. Whereas most organisations prefer to simply migrate their mailboxes into the cloud and be done with it, some choose a hybrid configuration, where some mailboxes are on premises and others in the cloud.
Be careful with the deployment assistant site: there are two of them. Most of the online tutorials will point you to the old deployment site.
In addition to being designed to provide advice about the previous (Exchange 2010-based) incarnation of Office 365, it doesn't work properly in IE10 or Firefox, requiring Chrome to actually get anywhere. The new site is far better coded.
There are really three options for migration: a staged migration, a cutover migration and a hybrid deployment. Hybrid deployments are for organisations with more than 1,000 mailboxes (the Office 365 limit) or those that have legal requirements for certain users' mail to be on premises. “Cutover” and “staged” are basically the same, except that cutover does it all at once while staged handles chunks of users in batches.
Microsoft has done a good job of making it simple
Performing the migration is straightforward. You start by making sure Outlook Anywhere – and the requisite Unified Communications Certificate – is configured. Use the Exchange Remote Connectivity Analyzer to verify this is all working.
On your on-premises Exchange server you need to give one user relevant rights. The magic bit of PowerShell I use is “Add-ADPermission -Identity [Mailbox Database] -User [Migration User] -ExtendedRights Receive-As”. This grants the “receive-as” permission to the migration administrative account I am using for the entire database accounts I wish to migrate.
You can also assign this on a per-user basis if you prefer. Receive-as allows the administrative user to poke his or her nose into the other Exchange mailboxes – a necessary feature if one is to migrate these accounts into Office 365.
Now you have remote connectivity to your Exchange server (Outlook Anywhere) and an administrative user who can access all the required mailboxes. A few button clicks in Office 365's interface and a very simple wizard and you are away.
Office 365 will create all the relevant users for you and keep syncronising the mailboxes with Exchange until you tell it to stop. This gives you time to test everything and cut the DNS over.
All in all, Microsoft has done a good job of making it simple to get onto its cloudy service. And if it isn't simple enough, third-party vendors have stepped in to provide more user-friendly tools for the task.
Migrating email is not the pain it used to be, and about time too. ®
In this Office 365 How to video, Trevor Pott walks us through Office 365 and email migration.
Sponsored: Becoming a Pragmatic Security Leader