Don’t adopt the Cupertino plan.

There are probably some sheepherders in remote areas of Uzbekistan who do not know that Apple’s new mega data center resides in the equally culturally arid desert of North Carolina. It will open to the usual hype and exaggeration next Monday. But, for the rest of us, that message has been broadcast loudly from the rooftops of Cupertino and points west for well over a year now.

First, let’s revisit what a robust, secure, distributed back-up plan looks like.

We start with the Pindelski Plan:

The Pindelski Plan. Two back-ups in the office – full and sequential. One in the car.
One somewhere else at an undisclosed location.

Now the Cupertino Plan:

Apple’s idea of multiple back-up sites.

So when Mobile Me(ss) next goes down, you will know why.

Do you feel lucky?

And in case you want confirmation of this easy target, here it is in Google Earth:

Yes, it will store a list of all your replaceable movies and tunes. But it’s probably the worst possible idea for backing up your precious photographs.

Wait a minute, you say. They will simply use the existing MobileMe (on the Hayward fault?) to back-up data. Uh huh. And you last used MobileMe when, exactly?

Finally uploaded.

After two weeks of uploading, some 170gB – comprising my user settings and Lightroom catalog – have been uploaded to CrashPlan’s servers. I first wrote about this cloud storage service here.

100%! Two weeks of uploading ….

Hereafter, uploads will be limited to incremental changes as new photos are added to Lightroom.

So how well does recovery work? Click on the ‘Restore’ icon and this is what you see (CrashPlan uses East coast time):

Note that you can recover at any date – just like in Apple’s Time Machine.

The first, unnamed, dropdown is the User’s directory, which CrashPlan uploads by default. The second, ‘HackPro HD’ contains only those directories elected when the upload commenced. In my case, that means my Lightroom catalog:

Drill down and you get to the catalog of Pictures:

One more step and you see the actual RAW, TIF or JPG files:

Check the files to recover and you see this:

I clicked on ‘Click here to download your restored files’ and the 12.5mB RAW file was deposited on the HackPro’s Desktop in 24 seconds, using my 10 mb/s broadband connection. That’s 0.52 mBytes/second compared to the theoretical maximum of 1.25 mB/s (10 megabits equal 1.25 megabytes and the line is 10 megabits/second). Not bad. The Desktop is a good destination as there’s no risk of overwriting your Lightroom catalog.

A couple of clicks and the file is in Photoshop, ready for processing:

File restored from CrashPlan’s cloud server. G1, kit lens @ 30mm, 1/320, f/5.6, ISO 100.

You can also restore the sidecar file with all the processing data, to avoid having to reprocess the picture.

Restored Zip files remain separately available for 24 hrs – the original RAW (or whatever) file remains untouched:

The only anomaly encountered so far is the wildly erratic reporting of upload status. I asked for daily email updates and got them at inconsistent times. That’s troubling, as inconsistency is the last thing I want from a cloud backup service, but the files appear properly uploaded and easily restored:

Erratic reporting from CrashPlan.

A check of files in my HackPro Lightroom and the CrashPlan directories confirms a like file count.

So, for $50 a year with currently unlimited data volumes, CrashPlan looks to be a useful supplementary backup plan. Just don’t make it your primary one. Are you about to trust all your pictures solely to a fragile ‘cloud’, where you have no independent verification of the adequacy of procedures or the financial solvency of the business?

Finally, recalling my earlier mantra that the only valid backup plan is one which has the qualities of paranoia and mistrust at its core, I made a reminder in iCal to prompt me to do a test restore of a file monthly. That way I get some comfort that the cloud database is not corrupted. It takes seconds to do.

Update June 13, 2011:

With incremental upgrades to my Lightroom catalog on the HackPro work machine being conferred faultlessly, and automatically, in the CrashPlan cloud storage database, I have signed up for 3 years for a total of $119.99:

In Part V I look at backing up additional files and at how best to backup applications.

Alternatives.

Cloud backups are not for everyone and, as I explained earlier, should never be your first line of defense against loss of your photographs. No one foresaw Fukushima happening. The power supplies ended up below the water line, useless. If it can go wrong it will and one of the great disasters to befall the world in the twenty-first century will likely be a massive failure of the ‘cloud’ through human error or enemy action.

What prompts this thought is an email from a friend of the blog who points out the futility of adopting CrashPlan for his very large photo library. A picture being worth a few words, here is his problem:

A reader’s problem.

I am a patient man but, even so, waiting for the best part of a year for your initial upload to complete is unrealistic. While CrashPlan does offer to provide you with a ‘seed drive’ by mail where you copy your pictures and which you mail back to them, this particular reader is in the UK where no such service exists. Eventually, this reader decided to upload smaller JPGs and not RAW files, so he will have a credible recovery source, albeit not of the highest quality afforded by RAW originals.

So here are some ideas, in addition to the JPG one, where your upload broadband speeed is slow and your picture catalog large:

• Identify the best pictures and upload first. In Lightroom the ‘star rating’ system makes it easy to create a separate catalog with just the highest rated pictures.
• Use the CrashPlan software to populate a friend’s computer if you can get comfortable with all the issues of trust, integrity and competence this approach raises.
• Adopt my CarBak approach, meaning offsite storage, and encrypt the data in case of theft.
• Don’t waste time backing up applications, movies (unless they are home movies) and music. You can always buy those again. And with Apple shortly migrating to cloud storage of all your movies and tunes, recovery will be easy as long as no one rains on their cloud. The problem with Apple’s cloud storage, unlike with CrashPlan’s, is that everyone knows where it is in North Carolina, open to attack. CrashPlan wisely keeps mum on the location(s) of its drive farms.

Getting there.

Details of CrashPlan, a cloud storage backup system, appear here.

After 5 days of chugging away, CrashPlan now reports as follows:

So at this rate I expect all 170gB of my pictures/data will be in the cloud in ten days’ time. Thereafter the rate of adding new snaps will be much lower, of course. That’s still within the 30 day free trial which should give me time to test incremental uploads as pictures are added and also to test the Restore function. It’s not 143,000+ pictures, BTW. Most of that count is system files in my User directory which CP uploads by default.

If all is well I will likely sign up for 12 months for $50.

I will report back in Part III.

A third line of defense.

When I recently wrote about CarBak, the backup portable hard drive with all my pictures on it which I keep in my car, it was not lost on me that the Big One (I live in Northern California, surrounded by major geological fault lines) would take care of the home and car in one mighty outpouring of energy.

The Hayward Fault …. and me!

Further, for terrorists there are only a few high value targets in the US. These include famous structures such as The Brooklyn Bridge, The White House and Congress in DC, Sears Tower in Chicago, The Transamerica Building and the Golden Gate Bridge in San Francisco. All simply fabulous, though for me the Brooklyn Bridge is #1 by a fair margin for its sheer beauty, comparable to anything, anywhere. So living near one of those is probably not a great idea if you want to keep local data safe.

At the time I added CarBak I had also determined to go with a cloud storage back-up plan but could not get comfortable with anything out there.

However, when Adam Pash at Lifehacker talks up anything I listen hard. You see, Mr. Pash was the person who inspired me to build the HackPro to replace Apple’s awful desktop hardware which had been nothing but a story of mechanical failure for this user for many years. That proves to have been the single best computer thing I have ever done, so call me a Pash Groupie. The cloud storage service Pash speaks about is named CrashPlan and I just signed up for a 30 day free trial.

The appeals of CrashPlan include:

• Unlimited volumes of data for $5/month
• Incremental timed back-ups
• 448-bit encryption of your data – you lose your password and you are dead meat as they do not know it
• Servers not located on the Hayward Fault, best as I can tell

While you can elect timed backups throughout the day and can even allocate the CPU percentage usage to the task, I cannot afford hardware bottlenecks during my working day. As my local, incremental, daily Carbon Copy Cloner backup (thank you Mr. Bombich) runs at midnight, and takes a few minutes, I set CrashPlan to run from 1am to 6am. Lower broadband traffic at that time helps, given the awful state of US broadband which is, for the most part, an order of magnitude slower than that in South Korea. Or even France! And because my provider – AT&T – is still running copper cable which was likely installed when Alexander Graham Bell was still having his nappies changed, I cannot get anything faster. They tried and my speed actually fell. Yup, that’s The Phone Company for you. America’s future.

So my backup plan now looks like this:

The only part of CrashPlan inside the HackPro’s case is their software. Search me where the data reside.

I have started with CrashPlan just backing up my User (login) and Pictures folders, the latter containing my Lightroom library where all my ‘serious’ snaps reside. (The family goo-goo stuff is in iPhoto), which figures as follows:

The eight days shown assumes full time availability. As I have gone to 1am-6am, that means it will likely take some 38 days for a first complete backup, meaning my 30 day trial will have expired. No matter. I’ll buy one month for $5 to finish and test the result.

CrashPlan prices.

CrashPlan also offers a free option which allows you to store data on a friend’s computer using their software. This is an idea comparable in stupidity to, say, the US invading China (comparable, but hardly impossible), and I can think of several million reasons not to do this. If your data is not worth $60 a year to you then it’s not worth anything.

Or, stated differently, the only viable approach to backup is to totally distrust everyone, especially yourself. If it can go wrong, it will. Did I ever tell you about the IT geek who worked for me whom I fired in a moment of righteous (self) anger? The son of an unmarried mother swore up and down we were doing daily backups. I checked the logs and they said we were. Then one day we had a server crash and had to do a restore. The backups were useless. They had never been tested …. he was lucky the windows did not open when I fired him. My anger was misdirected, of course. I had made the mistake of trusting him. The fault was mine.

Ever since then I test my backups monthly, making it a point to boot from the backup HDD in the HackPro and restoring a file or two from the Time Machine and CarBak backups. Once you make it a routine, it’s a few minutes and no big deal.

Finally, if you are in a big hurry to get your local data uploaded, CrashPlan will send you a hard drive for your copy and you mail it back. If you have no backup at all, this is the way to go. Hurry!

Making a cloud storage vendor your primary backup plan is back with the China invasion scenario for smarts. You cannot check financial stability (remember how safe Enron was? Can you spell AIG now in for $187bn of your money?) any more than professional auditors can. You will never know where the servers are located, the vendor using the shield of secrecy to refuse disclosure (for all I know they are in downtown Pyongyang) and there’s no one on earth you can believe about their security or the integrity of their employees. Statistically, some are likely to have criminal records/DUIs/spousal abuse/whatever on their copy sheet.

So forget about due diligence. A time sink.

And finally, the cloud is a fragile beast. Remember Bell and his nappies and the Telco’s wiring? How do you know the cloud will be available when you need it? The Big One will take out all broadband in your area if the terrorists don’t get there first. And it doesn’t have to be a terrorist. When I was working in San Mateo at the peak of the Internet bubble, a construction worker busy expanding the San Mateo Bridge in time for the bursting of that bubble managed to cut a huge bundle of optical fibers nearby and it took fully six months to reconstruct them. Broadband speeds plummeted, all because of one Joe Six Pack.

But as a third line of defense (OK, fourth line in my case) $60 or so a year per computer does at least compare to the vitamin you fool yourself will stave off next winter’s cold. It probably does nothing, but you feel good about it.

I’ll report back when I have uploaded all and tested a download, which will be in several weeks time.

Meanwhile I hope the Big One can wait until Day 39 hence, when all my snaps are on CrashPlan’s servers.