Key Factors To Consider When Creating A Media Storage System

Storing your data has always been important, but now you have a lot more options for data storage.

We’ve been seeing an increase in questions about storage lately, and our editorial focus has had an emphasis on storage this year. With that in mind, we have been collecting questions, and that has led to more questions that some of you might not have even contemplated yet.

We sincerely hope this story gives you some more insight about what your best options for creating a media storage system that stores your data efficiently, effectively and safely.

Media Storage System Factor #1: Speed


How much throughput does your media storage system need and for what video formats?

The data throughput requirements for a number of major formats are listed in Figure 1. Many formats have very minimal throughput needs, so just about any disk subsystem can handle all of the major consumer video formats with no issues.

How do you achieve that level of throughput?

There are many factors that go into determining the speed of a particular hard drive. The biggest considerations are spindle speed and areal density. A drive with higher areal density means that more data will be moving under the drive’s heads per revolution.

Interestingly, this means that larger-capacity drives often have higher data transfer rates than smaller-capacity drives. But it also means that you have to watch your numbers carefully – it’s rare, but not unheard of, for drives with high spindle speeds to have lower performance than some drives with not-as-fast spindle speeds but large capacities.

Don’t believe us? OK, let’s go over to one of our favorite benchmark utilities, SiSoftware Sandra. According to their collected benchmarks of physical disks, the fastest benched physical drive as of the time of our test is the Western Digital WD740HLFS, which is a 74GB 10K RPM drive. It operates at 105.28MB/sec. However, Western Digital also made the WD1600AAJS, 7200 RPM 160GB drive that is almost as fast at 96.621MB/sec. On the other hand, there’s also the WD740GD, also a 74GB 10K RPM drive, which can attain only 61.956MB/sec. So spindle speed is not necessarily an indicator of a high data transfer rate – there’s more to it than that.

The fastest drive of all the drives tested in this particular version of Sandra is a 64GB SATA SSD, which benches at 160.001MB/sec. (We’ll talk more about SSDs in a moment.)

To get more performance out of multiple drives, you can put those drives into a RAID… we’ll talk more about those a little later as well.

Reliability


What goes wrong with disks, and how I can work around that?

The operational principle of a hard drive is a set of platters spinning at a high speed, with tiny heads hovering very close to (but not touching) the surface of each platter. (A common analogy is that of a modern jumbo jet aircraft flying at full speed but only a few inches from the ground.) So, to pull that off, a hard drive needs to have platters with an extremely smooth surface and, ideally, no bumps from the outside world to cause problems. This is why your computer’s hard drive should never be opened; even the tiniest smoke particle that landed on your hard drive’s platter could be disastrous. This is also why hard drives include small filters to clean the air coming into the drive. Ideally, your computer should also never be moved when it’s on (this also means that you should really close your laptop’s lid before picking it up).

Is there a way I can get higher performance out of my drives, and maybe added data protection if everything comes out right?

The solution that you are looking for might be a RAID, which stands for Redundant Array of Independent Disks. This describes a technology that allows multiple drives to be addressed together to show up in on the computer as a single volume. The “redundant” part of the name is a bit of a misnomer, though, since there are some configurations where there is actually no redundancy. See Figure 2 for a quick outline of RAID levels.

It is extremely important to note that using a RAID is not a substitute for keeping proper backups. You still need to make backups of your most important files.

Do RAID controllers matter?

Yes. While many motherboards out there include RAID functionality, it is often not of the same caliber or flexibility as that available from third parties. Not only can a third-party controller allow you to use different types of drives (e.g., if you wanted to use SCSI, SAS or Fiber Channel drives, you could probably find a controller that would let you do that), they generally include their own processors to free up your computer’s processor. This is especially important if you are running a RAID 5 or RAID 6.

That said, if you want to put together a simple stripe or mirror set of SATA drives, the performance of the RAID controller on the motherboard is generally pretty decent. It’s a good way to dabble in the advantages of RAIDs, to help you decide whether you want to go ahead and spring for the fast controller and enterprise-class hard drives.

Longterm Storage

Is it possible to store a hard drive for long periods of time without having bad things happen?

We’re not really sure, and that is something a lot of people are very curious about. Over long stretches of time, it is possible that the lubricant for the drive motor may leak out onto the platters, which would destroy both the motors and the platters. If you’ve got a really critical project, using redundant formats, e.g., writing the same project files to several different formats (or even extra copies onto the same type of media) would be very wise. You might consider writing to a USB Flash drive or Blu-ray Disc (or both!) in addition to a hard drive.

Are drives sensitive to temperature and humidity?

Yes, very much so. Like any other type of media, it’s recommended to keep anything that stores data in a range of comfortable temperatures. Shoot for 68°F at 40% relative humidity, but, in general, if you’re comfortable, your drives (and other media) should be fine.

Do SSDs offer any advantages when it comes to stray magnetism?

Yes. The way Flash memory is written does not involve magnetic fields, so theoretically nothing should cause the premature erasure of any data on an SSD. Long-term, however, we are curious to see the effects of time on the data stored on SSDs and other Flash media.


SSDs

What’s different about SSDs? What are the advantages and disadvantages?

We’ve been bandying the term SSD around a lot lately. SSD stands for Solid State Disk. Since Flash memory has gotten fast enough to be used like a hard drive, someone decided to try doing just that, and the result is the SSD. Think of it as a faster version of the ubiquitous USB Flash drive that is attached to your computer in a more permanent way.

Some advantages are the aforementioned resistance to stray magnetism, generally high read speeds, very minimal (if any) heat generation and generally lower power consumption (particularly compared to the power consumption of high-performance desktop hard drives).

However, SSD-land is not a paradise world. Writes tend to be slower, because there is frequently an erase cycle involved before data can be written to an SSD. SSDs, therefore, aren’t well-suited for video capture that’s much more advanced than the most popular consumer formats.

Also, don’t try to defragment an SSD. Because everything on an SSD is accessible at the same speed (unlike on a hard drive), fragmented files will not take any longer to be accessed. In fact, you will shorten the life of any Flash memory device that you attempt to defragment. However, don’t forget to defrag your regular hard drives and disk arrays.

Enterprise-Class Drives

What makes enterprise-class drives worth considering?

We would recommend enterprise-class hard drives to anyone who doesn’t mind paying a small premium for a better-engineered hard drive that will generally have higher performance than a desktop counterpart. A typical enterprise-class hard drive has a spindle that is anchored on the top and bottom of the drive, instead of just the bottom of the drive. This makes the platters significantly more stable. These drives also include more-robust error correction and are designed to be operated 24 hours a day. (Most desktop drives are designed to run for a 40-hour week.)

It is important to note that many RAID controllers are designed to work best with enterprise-class hard drives, mainly because of the way they handle read errors. Left to its own devices, a desktop drive will keep trying to read the bad spot forever (and causes other things to freeze, by extension), but an enterprise drive knows the show must go on. It reports the error to the operating system and proceeds to the next operation.

What’s the price difference?

Entry-level enterprise (3-1/2″, SATA, 7200 RPM) drives run from 16 to 26 cents/GB; desktop drives run from 9 to 18 cents/GB. Other interfaces, such as SAS and Fiber Channel are also available, at a premium (of course). Enterprise-class drives are generally where you find drives that have faster spindle speeds, and there are some smaller form factor drives (e.g., notebook-sized drives) available as well. All told, no matter how you slice it, hard drives still deliver the most reasonable cost per gigabyte.

Optical Storage

How long do burned optical discs last?

We’ve heard numbers that range from two years to 50 years, so we honestly don’t know who to believe. For serious projects, it seems wise to use archival-grade media, as opposed to the el-cheapo media that you’d be using for test discs. So both the archival-grade and el-cheapo media have their places.

Any advantages to gold discs or specific dye types used by each manufacturer?

It depends upon whom you ask. Gold in general sounds like a wise idea, as it is more resistant to oxidation over time.

We aren’t sure about how the different dyes that are available will last over time. Each manufacturer claims that theirs is the best, of course. If it’s an important project, consider using discs from two different manufacturers (e.g. make sure your disc-burning program reports two different brands of discs, since the companies that market blank discs often simply buy discs from manufacturers and brand them however they please).

Does write speed matter?

Theoretically, yes. The argument is that, if the disc is spinning more slowly during the burning process, the write laser can be modulated more carefully by your computer’s disc burner, leading to a disc that will be more stable over time. If you have time to write a disc at a low speed, it might be time well-spent.

Nathan Beaman is an Apple Certified Final Cut and Motion Graphics Trainer.

1 COMMENT

  1. I'm a hobbiest video maker – no secret block busters to hide away – and I use Carbonite in addition to a desk top hardive for back ups.

     

    What are your thoughts about on-line back ups like Carbonite?

     

    Douglas J. O'Brien

    Jacksonville, Florida

LEAVE A REPLY

Please enter your comment!
Please enter your name here