03 May 2010

Using MobileMe's iDisk as an interim backup while traveling

Introduction

I use an Apple laptop hard disk as my primary (master) data storage device.  To provide interim backups while traveling, I use Apple's MobileMe iDisk for network backups to supplement primary backups only available to me when I'm at home.

Having dabbled with iDisk for a few years, I have two key constraints for using iDisk:
  • I don't always have a lot of bandwidth available (e.g., a mobile phone GPRS connection) and I don't want a frequent automatic sync to hog a limited connection.
  • I don't trust MobileMe with primary ownership of data or files.  Several years ago I switched to using the iDisk Documents folder (with local cache) for primary storage but then had several files magically disappear.
I've now evolved to using iDisk as a secondary backup medium.  I manually run these steps when I have plenty of bandwidth available.  There are two steps to this:
  • rsync files/folders from specific primary locations to a named directory under iDisk
  • Sync the iDisk
How to do it

The rsync command I use looks like this:


for fn in Desktop dev Documents Sites; do
   du -sk "/Users/my_username/$fn" | tee -a ~/logs/laptop_name-idisk.rsync.log
   rsync -avE --stats --delete "/Users/my_username/$fn" "/Volumes/my_mobileme_name/laptop_name/Users/my_username" | tee -a ~/logs/laptop_name-idisk.rsync.log
done

The rsync flags in use:

-a         archive (-rlptgoD no -H)
           -r    recursive
           -l    copy symlinks as symlinks
           -p    preserve permissions
           -t    preserve times
           -g    preserve group
           -o    preserve owner
           -D    same as "--devices --specials" (preserve device and special files)
-v         verbose
-E         preserve extended attributes
--stats    detailed info on sync
--delete   remove destination files not in source


Explanation:
  • I'm targeting specific locations that I want to backup that aren't overly big but tend to change frequently (in this case several folders from my home directory: Desktop, dev, Documents, Sites)
  • A basic log is maintained, including the size of what is being backed up (the "du" command)
  • I use rsync rather than copy because rsync is quite efficient - it generally only copies the differences, not the whole fileset.
  • The naming approach on the iDisk allows me to keep a backup by laptop name allowing me to keep discrete backup collections over time.  My old laptop and backups sit beside my current laptop backups.
  • The naming approach also means I don't use any of the default directories supplied by iDisk as I'm not confident that Apple won't monkey with them.
  • ~/Library/Mail is a high change area but not backed up here (see below for why)
The rsync updates the local iDisk cache.  Once the rsync is complete (after the first rsync I find it takes less than 10 seconds for subsequent rsyncs), manually kick off an iDisk network sync (e.g., via a Finder window, clicking on the icon next to iDisk).

An additional benefit to having a network backup of my important files and folders is that I can view and/or edit these files from the web, iphone, or PC.  I find that being able to access email/IMAP from alternative locations is the most useful feature, but I have had minor benefit from accessing files as well when my laptop was unavailable or inconvenient to access (e.g., quick check of a contract term in the back of a taxi on an iphone).

Other Backups

I have two other forms of backups:
  • Irregular use of Time Machine to a Time Capsule, typically once a week if my travel schedule permits.
  • MobileMe's IMAP for all email filing (and IMAP generally for all email).
Basically, if I'm traveling, I rely on rsync/iDisk and IMAP for backups.  I also have the ability to recover a whole machine from a fairly recent Time Machine backup.

Success Story

In June 2009 I lost my laptop HDD on a return flight home after 2 weeks of travel.  I had a Time Machine backup from right before I'd left on travel, and occasional iDisk rsyncs while traveling.

Once I got home I found an older HDD of sufficient size and restored from the Time Machine image from the Time Capsule.  This gave me a system that was just over 2 weeks "behind".  Once IMAP synchronized my mailboxes, that only left a few documents missing that I'd created while traveling.  Luckily I'd run an rsync and iDisk right before my return flight, so once I'd restored those, I'd recovered everything I'd worked on over the two weeks of travel, only missing only some IMAP filing I'd done on the plane.

Weakness

The primary flaw in my approach is that you have to have the discipline to remember to manually kick off the rsync and iDisk sync after you've made changes you don't want to lose.  I certainly don't always remember to run it, nor do I always have a good Internet connection available to enable it.  However, I find that remembering sometimes is always better than not having any recent backup at all.

Alternative Approaches

An obvious alternative is to use the MobileMeBackup program that is preloaded onto your iDisk under the Software/Backup directory.  Using this tool, you should be able to perform a similar type of backup to what I've done here.  I've not tried it as it was considered buggy back when I first started using iDisk for network backups.  I'll likely eventually try this and may shift to it if it works.

A viable alternative approach is to carry around a portable external hard drive, and make Time Machine backups to it more frequently than you would otherwise do over the network via iDisk.  You could basically keep a complete system image relatively up-to-date if you do this.  More hassle, but lower risk and easier recovery if your primary HDD fails.  However, if you get your laptop bag and external HDD stolen, you'll be worse off.

While on holiday recently, I was clearing images off of camera SD card memory as it filled up.  I put these images both on the laptop HDD and an external HDD.  This protects me from laptop HDD failure, but wouldn't help if both the laptop and external HDD was stolen.

iDisk Comparison to DropBox

DropBox is a popular alternative to iDisk.  I find DropBox to be better at quickly and selectively sharing files, it has better cross-platform support (particularly with a basic Android client), and it's sync algorithm seems to work better than the iDisk equivalent.  You could certainly do everything described here with DropBox.

The downside with DropBox is having to pay $120 per year for 50GB of storage versus $60-100 per year ($60 on promotion, e.g., with a new Apple laptop; otherwise $100) for 20GB of storage with MobileMe.  I find 20GB to be plenty for IMAP, iDisk and photos providing I filter out big auto-generated emailed business reports (store on laptop disk not in IMAP), and only upload small edited sets of photos.  I'll probably exhaust the 20GB in 2-3 more years at my current pace, but I'd expect Apple to increase the minimum by the time I would otherwise be running out of space.

MobileMe is of course more than just iDisk, so if you use more of it's features, it increases in value relative to DropBox.

Both iDisk and DropBox are usable choices, the differences are not sufficiently material to strongly argue for one or the other.  I have seen iDisk improve over the last few years and I'd expect Apple to eventually catch up with DropBox.

Conclusion

While I'm not confident in using MobileMe's iDisk as a primary storage location, I have found it useful as a network backup.  Combined with normal backups using Time Machine and Time Capsule, it provides a high-confidence recovery from damaged or lost primary use laptops.

2 comments:

  1. You could create a "connection quality test" at the beginning of the script and if your internet connection is present, and of sufficient bandwidth, automatically backup once an hour or something reasonable.

    To test connection quality, I'd suggest timing three consecutive downloads of increasing sizes, if the first fails to complete in a reasonable amount of time, don't continue testing...you get the picture.

    ReplyDelete
  2. I had a play with "Backup" that comes with MobileMe (iDisk/Software/Backup/MobileMeBackupv3.2). I'm not compelled to use it.

    It puts files into a "package" in the iDisk/Backup/Name.TDStamp.FullBackup. While you can open the package (Show Package Contents), the backed up files aren't directly available, they are in a proprietary (not folders/files) backup format. Therefore you can't conveniently find files via an iphone idisk client as you can with the method above. This approach does serve the purpose of making minor backups while traveling.

    MobileMe is already backing up Address Book, Calendar and other "systems" oriented files by virtue of synchronization (assuming you're doing that), so using Backup is superfluous in this sense.

    I've left Backup acting on Address Book and Calendar (only), but for no particularly good reason (Just Another Backup).

    ReplyDelete

Note: Only a member of this blog may post a comment.