 |
 |
 |
 |
Moving a Petabyte of Data
|
In a recent speech I asserted it was faster to send a petabyte of data from San Francisco to Hong Kong by sailboat, than by the Internet. I got quite a few "how can that possibly be true?" kinds of questions, so here's the math. (Full disclosure, I am a mathematician by training, which guarantees me a lifetime of small "off by one" errors in all subsequent calculations. So if I get something wrong, be gentle). A petabyte is a thousand terabytes, which is a million gigabytes, or a billion megabytes. Or eight billion megabits. With me so far? So if you had a half megabit per second Internet connection, which is relatively high in the U.S. (although relatively low compared to residential bandwidth available in, say, Korea), it'd take you 16 billion seconds, or 266 million minutes, or 507 years to transmit the data. Can you sail to Hong Kong faster than that? At a full megabit, just divide the time in half. Even at 100 megabits (about the highest, generally available, of any carrier I've seen), it's only a few years. As Hal Stern once said to me, "Never underestimate the bandwidth of a station wagon full of storage driving down the Jersey Turnpike." And now you understand why tape-based storage has such a lasting appeal to so many enterprises for recording, compiling, transporting or just plain archiving very large quantities of data. From video surveillance to trading data, standard tapes are 500GB each (currently) and fit nicely into cardboard boxes with overnight express labels. Another big benefit to tape as an archive format: When the data's at rest, it consumes no electricity. Just imagine a petabyte of data spinning on even the most power efficient disk storage. (For reference, a petabyte of active disk-based storage is the equivalent of more than 40 Thumpers, each drawing more than a kilowatt and tipping the scales at something north of 150 lbs, slightly tougher to put on a sailboat — or in an overnight envelope). For data to be available, disks have to be kept spinning and cool. Tape has no equivalent requirement. Now there is no one hammer for all nails, and tape isn't perfect for a lot of applications (near line storage, etc.). But it plays a prominent role in some remarkably cutting-edge, high-performance computing applications, along with social networking and content aggregation sites (who think nothing of gathering terabytes of data every day). Tape archive isn't just for banks or telcos running mainframes (although we're good there, too). So yes, at least for now, it's faster to send a petabyte of data via a sailboat than the Internet (at least defined by the bandwidth to which most of us have access). That's another reason we're refreshing our Solaris on DVD program. It's more efficient for many folks to get a 4 Gigabyte DVD in the mail (for free) than nurse our download centers a megabit at a time. (And I apologize for how slow the DVD deliveries have been; we haven't exactly executed perfectly here, but hopefully it's getting better as I type.) And I don't want to even think about moving a zettabyte. Jonathan Schwartz | |||||||
 |
