Best video archival method

[futball8]

Hey all,

Perhaps this has been addressed previously, but I had a hard time finding anything in the forums.

Context of question:

I work for a local cable channel, producing video content – around 15-20hrs of content per month.

All video is shot on SDHC cards – completely tapeless workflow.

I’ve already abandoned the idea of archiving all my raw footage – it would just be too much, however I obviously do need to archive my final programming edits. These files are MPEG2 format – the compression I use works out to approximately 2GB per hr., so I have 30-40GB per month that I need to archive. I started to backup todata DVDs, single-layer first and thendual-layer, but ittakes so long to burn and backup this way, I kind of gave up.

So, that brings me to my final quesiton:

Would it be safe for me to just do archival backups on normal 1TB external HDDs? I’ve heard that this is not safe, because alldrives are destined to fail at some point. I undertand this, but what if the drive was rarely, if ever used? I dump 1TB of data on it and it gets filed away and accessed maybe a couple times a year.

What about Blu-ray? I see the cheaper brand media can be bought for around $5-$8 per 25GB disc. This would obviously be cheaper per GB than to use hard drives, but are there disadvantages and what about data integrity?

Anyone with experience in these issues? Many thanks in advance!

[Rob]

I wouldn’t rely on a hard drive. In an experience I had, a hard drive of mine was used daily. It was always on and worked flawlessly. Then I got a larger one and safely stored this one. When I went to use it again a few months later, it didn’t work. Mounting was hit or miss. When it finally did mount, data would seem to be corrupted shortly after a file transfer to the drive. I tried to repair the disk using Disk Warrior, but then the disk just wouldn’t mount again. It seems it went bad because it wasn’t in its daily use anymore. Doesn’t really make sense, but I NEVER had a problem with the drive when it was used every day.

I really don’t recommend relying on a hard drive, but if you must, back up to 2 or 3 hard drives. Actually, no matter what medium to back up to, you should have 2 or 3 copies.

Whether it’s DVD or Blu-ray, I’ve always felt data discs were very reliable. If the disc is burned successfully, that data isn’t going anywhere as long as you keep the disc safe, which isn’t hard to do. Just keep it in one of those CD book things.

I’d suggest LTO tape, but you’re not backing up enough data for it, and initially it’s expensive. But certainly keep LTO tape in mind for the future. An LTO machine can cost as little as $3000, but the 800GB tape cartridges are only $45. This is a very reliable and speedy medium for archiving if you are generating enough content for it to be worth the investment and if you’re getting paid enough.

[XTR-91]

“I wouldn’t rely on a hard drive. In an experience I had, a hard drive of mine was used daily. It was always on and worked flawlessly.”

If you take proper care in working with and moving around, an external hardrive should be just as reliable as the one inside of your computer. If you’ve purchasedan un-branded (China merchandise) from eBay ora terradrive for under $100, then that’s probably the reason for having storage as its most unreliable state.

My experience with CDs, DVDs, and Blue-ray discs (with the exception of hard coating) are prone to tears and scratches just by the nature of basic use. Files storage on discs with my experience is far less flexible in terms of writing and re-writing. CD-R and DVD-R discs can be written only once before they are sealed and finalized. CD-RW and DVD-RW are designed in a specific way to be rewritable as storage, but still have their limitations. After a certain number of writing sessions, the disc must be eraced for any further use. Files cannot be deleted individually. I’ve never had any experience with Blue-ray or any of its characteristics. DVD-RAM is another option (if your drive supports it) that allows a significantly greater amount of writing sessions (somewhere around 10,000,000).

Personally, I’d go with an expensive hardrive with 1 or 2 TB. Most are around the $100-$100 price range.

[Rob]

“If you’ve purchasedan un-branded (China merchandise) from eBay ora terradrive for under $100, then that’s probably the reason for having storage as its most unreliable state.”

Nah, the drive that died on me was a 250GB LaCie drive. They are certainly a respected brand.

[composite1]

This is a pretty serious subject. As always, in the mad dash to incorporate the latest tech funky little things like ‘archiving’ got overlooked when it came to tapeless workflow. Don’t feel bad. Everyone from the big studios to guys like yourself are all trying to figure out how to do this.

Right now your bets are: Oldschool tape back-up, Optical media (DVD, Blu-Ray) and harddrive. Harddrive is king at the moment because of the large storage capacities but as Rob mentioned it’s not 100%. Even with the new SSD’s there seems to be a problem with data volatility in the long term. DVD’s and Blu-Ray are reliable, but the amount of disks needed to back up original or uncompressed material can get tedious and expensive. Oldschool tape backup is the safest, but the technology has moved away from it’s use and keeping equipment around to play it will eventually become impractical.

The ‘plan’ for now is to wait for solid-state media to reach a level of expense to where it is cheap enough to use as videotape or DVD’s are. I’m not particularly fond of that idea because the US and World Economies aren’t working off the ‘things getting cheaper’ mode anymore. So for now, I’m using a combo of harddrive and tape backups. Backing up to 2 or 3 harddrives for me is impractical and a redundant expense. I suggest for those of you backing up to harddrive to do regular ‘run up checks’ on your archival drives and basic maintenance as needed to keep them in working order. The big studios and networks are doing the exact same thing so I don’t see why it wouldn’t work for you.

[XTR-91]

I could probably safely conclude that external hardrive storage is always becoming more safe and practical. 250 GB drives seem to be disappearing from most site sellers that advertise the most recent (currently selling) technology. The current hard disk drive solutionsare coming along more rugged and reliable. I do agree with Rob’s assessment, however, that optical media provide the safest mode of storage when thoseinvoluntary things happen, such as crashes.

[Anonymous]

Hard drives fail, I have three of them sitting not far from my desk. I’m currently using DVD’s until the cost of SDHC cards comes down to where they are affordable to just archive the card.Also since your really only archiving your final projects maybe it would make senseto spend the money and back them up on the SDHC cards, they may cost a bit, but they are practically indiscutable, and I ask how important is you project not to spend a little extra money?

If you have to use a hard drive definitelyuse multiple drives, or a raid, I’d look at getting a Drobo.

Home

[Rob]

We have a Drobo at work. It’s a nuisance. It doesn’t always mount to Drobo Dashboard. Loading data is REALLY slow when it’s close to full (i’m talking hours to transfer a few gigs) and Drobo Dashboard isn’t very accurate when displaying the amount of space left. Ours currently says 700GB left, and we’ve been having trouble transferring more data to it. Turns out it’s actually completely full. So we wasted about 2 weeks trying to figure out what the problem was.

Drobo is also expensive. With dual disk redundancy, our Drobo only has about 10TB of available space even though we have 8 2TB drives in it.

I haven’t really decided if its really worth $6000.

[Anonymous]

One of the problems with magnetic storage (Hard Drives), is that the magnetic fieldsweaken over time. This will cause data bits to be lost. One wrong data bit in a file will invalidate the whole file. The only way to maintain them at the optimum, is to read and then rewrite the data from time to time.

[Author]

Posts