You are here

How do you store and access home video?

dawgcatcher's picture
Last seen: 1 month 3 weeks ago
Joined: 02/08/2012 - 4:26pm

Hey everyone

Im new to this group, so please forgive me if this subject has already been covered or if it is too elementary for this forum. But, I need your help

I am an entry-level/intermediate video enthusiast and I would like to store the footage from the 100 or so miniDV and older Digital8 tapes in such a way that I can:

1. access the footage for use in Final Cut without having to capture from the tape each time.

2. access the footage from my TV and scrub through it and access it randomly (i.e. pull up any home video and watch it).

3. insure that it is saved securely

While this seems simple enough, I still have problems wrapping my mind around the very best way to do it. The time involved with ripping this footage into wherever it will end up will be substantial, so I want to make sure I do it in the very best way possible.

What file organization method should I use to catalog the footage?

Is there a 3rd party software that helps with this? Maybe a plug-in for FCPX? Do you have to simply batch capture each tape to the HDD?

What file type should I use (AVI, MPEG, etc.)

Should I use something like Apple TV to view it on my TV and will that automatically convert the files so they can be viewed?

Should I invest in a HDD RAID system? How much storage will I need per hour of uncompressed SD video? Of AVCHD video?

Back-up system?

What is the best way to connect to this footage from my MacBook Pro for editing?

How do you interfacethe footageto your home entertainment system router?

Ideally, I would like to be able to choose composite 3 or something like that as a TV input and have a file system pop up and I could easily navigate to the exact footage for which I am searching, and watch it with my family. Better yet, figure out a way that is simple enough for my wife or kids to pull it up. All the while being able to open FCPX and ingest any footage into the interface for editing. I am less concerned with the cost of this than with the simplicity and functionality of it once it is set up. We have so many memories locked up in these tapes, and I need to find a way to relive them conveniently.

Any help would be greatly appreciated. THANKS!

Jack


Ed Rogers's picture
Last seen: 1 day 6 hours ago
Joined: 02/02/2008 - 2:35am
Plus Member

Dawgcatcher,

This is at least a two part answer. First, I'm assuming you haven't edited or created DVD or Blu-Ray discs of the tapes? If not, a RAID system would be the best, especially if you're going to edit.

File location is crucial. It'll keep you from going insane, trying to locate stuff that's just been haphazardly placed. Myself, I have each project named, sometimes include a date. I say "Project" because sometimes projects have more than one tape. Within the project folder I usually have these standard sub folders: video, photos, graphics and music. In my editing software (Adobe Premiere CS5), I have matching bins - I also have another bin labeled Titles. This, my friend, will keep you sane. File format is basically up to you. My general rule is compress as few times as possible. While this will mean more HDD real estate, it'll keep your footage as fresh as the day you made it.

Importing your footage will require a capture device. There are many on the market and I am not too certain what the best platform for Macs is. I use a Matrox RTX2, coupled with a DVD/VHS/ player,for my desktop. Matrox has other external devices: http://www.matrox.com/video/en/home/ I don't own them, so I can't make any recommendations.

The final part is backup. I can't stress enough how importanta backup is. It can be HDD, DVDs, blu-Rays ormultiple redundancy. I useDVDs and Blu-rays to watch with my family, they serve as one back up. I also have external drives that I back up to - especiallyincluding my editing files.

Hundreds of tapes is a lot of data to store, I'm sure you'll be adding to it. Purchase as much storage as you can afford. I hope this helps!


SafeHarbor's picture
Last seen: 1 month 1 week ago
Joined: 01/22/2010 - 3:28pm

Hi Jack,

DV footage, captured via 1394 Firewire, is an exact copy of the data on the tape, and uses about 13GB per hour, so you should get 70+ tapes on a 1TB drive. For backup, you can use a second drive, or get a RAID 1 system which will have TWO drives in it and automatically mirrors one drive to the other for full-time backup. A 4TB RAID 1 unit should handle your entire collection, including backup.

To keep track of it all, consider CatDV software from SquareBox. NASCAR uses a version of this to keep track of their entire archive of video from all races! Standard version is just $99 but quite powerful.

There are a ton of different ways to set the footage up for viewing on TV. You may want H.264 converted versions for playback using various devices like Apple TV, etc. You'll want to experiment a bit to find the best file size/quality setting that works well with your intended viewing hardware.

You mention "uncompressed video" but in reality ALL video from camcorders always has some type of compression applied. Trust me when I say you do NOT want to deal with uncompressed, stick with the native DV or convert to an H.264 format for convenience. You'd need a Blackmagic or Matrox capture device to even get uncompressed capture from tape anyway, and to no advantage since the source was already compressed to start with, nothing to gain.

Hope this helps to point you in the right direction, as there is no ONE way to do this, each user will have different ideas about what is best for their needs.

Thanks

Jeff Pulera
Safe Harbor Computers


Rob Grauert's picture
Last seen: 7 months 2 weeks ago
Joined: 02/16/2008 - 10:47pm

1. access the footage for use in Final Cut without having to capture from the tape each time.

You'll want to use Final Cut to capture the tapes because Final Cut will convert them to .mov files, which is what Final Cut wants.

2. access the footage from my TV and scrub through it and access it randomly (i.e. pull up any home video and watch it).

Off the top of my head, I'm thinking you'd compress the videos to .mp4 and bring them into iTunes. Then use Apple TV to access them. I THINK that would work.

3. insure that it is saved securely

Redundancy is always (and has been) the key to security. You need your files readily accessible at your computer, then you load the same files to an external hard drive and tuck it away somewhere in your house, and finally you have those same files on another hard drive off site incase there's a fire or something.

What file organization method should I use to catalog the footage?


How you choose to name and organize your media is typically dependent on the media itself. You have to figure out what works best for you based on how you want to use your media and what information you want readily available if you're looking for something in particular. I generally name a file based on the content of the footage and the date the footage was shot.

Is there a 3rd party software that helps with this?

No. You have to take the time and do it yourself. No software will read your mind to figure out how to best organize your footage.

What file type should I use (AVI, MPEG, etc.)

<span>You're using Final Cut. Capture the tapes as .mov files encoded as ProRes. Those will be your digital Masters.</span>

Should I use something like Apple TV to view it on my TV and will that automatically convert the files so they can be viewed?

<span>I don't think Apple TV does any converting for you. I could be wrong, but I'm sure the purpose of the device is to stream content from iTunes to your TV. You need to compress your digital Masters to something that is iTunes-friendly, which is .mp4</span>

Should I invest in a HDD RAID system?

<span>Not a bad idea. It doesn't sound like you need to go crazy with a RAID. A G-RAID connected via eSATA with a x4 eSATA card will provide enough bandwidth for smooth playback.</span>

Back-up system?

<span>Your backup drives don't have to be RAIDs. Standard external drives will be fine. Just make sure they are large enough.</span>

How do you interfacethe footageto your home entertainment system router?

<span>What does it mean to interface footage? Do you mean interlace?</span>

<span></span>

<span>Understand that what you want to bring up on your TV will be different from what you want to bring into FCP and edit. The stuff you edit in FCP should be the digital Masters captured as ProRes. The stuff you view on your TV should be your digital Masters compressed to .mp4. The Digital Masters live on your RAID and backup drives. The .mp4 only have to live in your iTunes folder.</span>


dawgcatcher's picture
Last seen: 1 month 3 weeks ago
Joined: 02/08/2012 - 4:26pm

Man, you guys are awesome! You guys need more kids or something else to occupy your time! THANK YOU so much for taking the time to give me such comprehensive answers! I have taken all your commentsto heartand I am still formulating what to do next.

My videos are basically divided up into:

Hunting - Deer

Hunting - Turkey

Hunting - Other

Skiing

Family

Other

And not necessarily in that order :) They all of course have dates associated with them. I figured this would be a subject that lots of folks have come accross. I guess I will end up with two copies of the footage - one in either the native Dv or MOV and one in the MP4 format that iTunes can digest for viewing on our family TV.

@Rob - When I said interfaced with my TV, I mean what will the screen look like that popps up on my TV that will allow me to search for and access these different videos. Also, I just bought FCPX, and I am learning it now - I love it compared to Avid Liquid...It reminds me of the Syudio products that I used to use. My computer supports the Thunderbolt connection, and there are some pretty cool HDD's available, but they are $$$$ http://store.apple.com/us/product/H5184VC/A/Thunderbolt. I still need to figure out how my TV talks to my RAID (or HDD) system.

@Jeff - thanks for the simple calculation on memory requirements - big help!

@Ed - your comment is my biggest hurdle - how to catalog all the footage - thanks for your advice.

I guess I'm looking for a "Here's what you do... It's the standard for what you are trying to do." :)

You guys have a great weekend! I look forward to reading more posts.

Jack



Woody Sanford's picture
Last seen: 4 days 15 hours ago
Joined: 10/19/2010 - 7:33am
Plus Member

dawg,

The compromise is what is the priority. Do you want a master to edit or something for playing on the TV? Its a format/delivery conflict. I don't think there is a simple way around this one yet, I say yet cause you never know what is down the road.

I'm a PC guy and use Adobe software. I use externals as volumes and catalog much the same as a library does. I use Adobe Bridge to manage my way through it. However, its completely something you have to set up and maintain. I couldn't tell you what is best for mac but I don't see anything that would get you away from having to set up your own filehierarchy system.

I'm not sure if you would want to have the same file accessible for both avenues anyway. There is always a lot of dead footage you would fast forward through if you went straight from the camera to the TV, so there would always be two products to work with. One master for editing and one for recreational viewing. Just make the later in a more friendly format for that.

I upload most of mine to Youtube and Vimeo after there has been some editing. I can send links and watch it on my TV if I want and so can family far away. Its really not any different once you go to cameras with cards, just then you are stuck archiving on drives but the transfer is faster.

I can't advise you on apple TV and ITunes but I don't think you are going to find a "Do all" system with the same format and really you would want some edit on your TV media anyway. Its kind of the nature of the beast so to speak.


dawgcatcher's picture
Last seen: 1 month 3 weeks ago
Joined: 02/08/2012 - 4:26pm

W, thanks for your comments, and I was afraid of that (needing 2 formats). As for the dead video, I was just hoping not to have to watch and selectively capture al the footage. I figured I could just scrub through it later...but I totally get your point. Thanks!

Jack



barnacle's picture
Last seen: 1 month 3 weeks ago
Joined: 09/21/2011 - 11:41pm

External hard drives are getting cheaper and cheaper. I have two external 1TB hard drives.

One keeps raw video and pictures. The other stores videos that have been edited in Adobe

Premiere Elements 8. For the pictures I have yearly folders. Inside each of those folders

is a file called YYMMDDpics, which is the date I uploaded the pictures. As for the video

clips I usually make a folder for each topic (eg Reno052012, Aliciabday2012, etc). This

system works for me and might give you some ideas. Although I haven't tried it, one

of the 1TB extgernal drives (MyPassport by Western Digital) only uses USB for power

and could probably be viewed on my HDTV. The drive cost $89 abouta year or so

ago.

I'm not a pro, just a home video shooter/editer.


birdcat's picture
Last seen: 1 year 12 months ago
Joined: 10/21/2005 - 10:09am
Plus Member Moderator

Having lost a bunch of video due to two hard drive failures in a month, I now do this:

1) Burn final output to DVD or BD (two copies)

2) Store a version in the cloud (Pick one: Vimeo, YouTube, my own domains)

3) Store video version on PC (MPEG2, MOV, MP4, something)

4) For select videos, store MP4 on phone

Bruce Paul 7Squared Productions http://www.7squared.com


artsmith's picture
Last seen: 3 weeks 3 days ago
Joined: 03/02/2011 - 9:06pm

'Dawgcatcher': I have had a similar problem, being currently engaged in a project which has occupied me for the last (almost) seven years. I haveconsiderably more than 100 DVD-equivalents of footage, mostly DVin 'standard' video, but more recently m2ts from a more recent model of camcorder. With the change to the new machine, I devised a new system, which takes account of the fact that my footage will, eventually, be shown in 1280 x 720 format with 'progressive-scan'. After each day's shooting, I download from the camcorder into my computer, the day'sshots. They go onto a hard-drive in 1920 x1080 format and are archived, exactly as-taken. Additionally, I sit down with a very good transcoding product and change all ofmy stuff to 1280 x 720p, vetting each shot individuallyand removing the first and last couple of seconds from each shot, (which has been allowed for), as that is where unwanted camera movement and unwanted sound, (due to camera-handling in low ambient noise conditions), is to be found. When the time comes for editing, I know then, thatI have footage I can rely upon; also, I am familiar with what is in each clip, 'for the present', anyway.

That might sound like a lot of work, but it only has to be done once, and I can, usually, see off more than 100 clips in an evening of uninterrupted work, more or less what I can'convert' in a batch overnight in my computer while I sleep. Each clip in the native format, m2ts, is auto-numbered. Those numbers are retained, basically and I use a small and simple renumbering programme to add a prefix, (based upon the date of shooting), and suffix (details of the file format). The numbering system begins with the date in a block of six digits 'yy-mm-dd'. That absolutely indentifies the day of shooting and generally arranges the contents of my files in date-order'. Using the original shot-number allows me to cross-index anything. My (Panasonic), software came with a very good item for keeping tabs on clips. If I want to view any particular clip for reference; if all else fails, I can always view the original clip in its 'native' 1920 x 1080 format, becausethe cross-indexing brings to the screen, the identical shot, to my 1280 x 720 conversion, except for format.

Vital, of course, as this 'grows' is a cast-iron indexing system. That is usually organised on the basis of 'location', not date, and records the first and last clip in eachday's 'shoot'. A typical clip number might be along these lines - 120713(161527)AVC720p. Mpg4 clips do not take up very much space on discs and the entirety of my shooting to date, 1700 clips, will fit onto less than the equivalent of two 'archive' DVD's, which will be put away, for safe keeping. This system leaves me with all options open as to what to do with the original m2ts clips, which are presently in secure storage as well.

For editing, I create a 'project' file into which everything which belongs to a particular project goes, under sub-directories, eg 'graphics', 'animation', 'music' (I compose my own) and so-on. For safety, the data is backed-up onto flash devices with a fair usable capacity, but it is not fed to the video-editor from there, for various technical reasons, but instead from a hard-drive. A back-up of the project fileonto the flash-device at the end of each day's work is probably a sound investment of time and effort.

As for how I am to show it all on our TV? I haven't a bloodyclue. I'll worry about that, when I have to.

Ian Smith - Dunedin, New Zealand.


BJBBJB's picture
Last seen: 5 months 4 weeks ago
Joined: 10/12/2011 - 3:06pm
Plus Member

This thread is very timely! I wasthinking about the back-end of this question (how to access the videos quickly via digital vs. disc media) just last week. The tips above were very helpful, other than I am a bit more PC and windows-centric and useProduction Premiere and Encore. But I have a similar mix of older analog and newer HDVtapes.

My first step wasto convert older analog say 8mm tapes to digital DV tapes to reduce any degradation as I got to editing them. Then I workthrough the footage, edit it down and create SD DVD's for theSD stuff and Blu-Ray forHDVusing Pr Pro and encore. Two copies, one gets filed away. Finally I use the Production Premiere project backup feature tobackup the rawvideo, assets, and project files to a blu-ray data disc for any future editing work on the project. I wish I had that backup feature years ago!

So I have a nice chunk of DVD's and Blu-rays already with lots more to do. Like the OP I was wondering about accessing this media "media-disc-free" if you will in the future. I don't use Apple TV. I know many devices (satellite boxes, TV's, BLu-Ray players etc.) show up in Windows as "Media Devices" so that is one thought.Put the media on a hard drive that is PC-accessible and access it through the media server if it can read the files.

So does ripping my already-made SD DVD's and Blu-ray's to MP4 make sense? Then I would need to "mount them" with a software DVD player wouldn't I?Not sure a media player could play them and keep the menus which I would like to keep. I would also like to keep my 5.1 sountrack (I use a 5.1 mix in Pr Pro at times, not saying my source is 5.1, but the disc is). However I think a media player could play the raw ripped video file without a menu if it was accessed directly?

Thenfornew videos produced, should my new workflow be to use Adobe Media Encoder to make a separate video file output in the MP4 or H264 format? Or would it make more sense to just rip the already-made DVD/BLu-ray and then "mount" that image somehow. That way you would keep the menus. Having the footage available without the menus might be a little messy but would be a fallback position.

Seems like the main decisions I have is to whether to rip to a DVD/Blu-Ray "image" type file/folder format or just get the raw video/audio out of the disc so it is playable.What file format to use. And what device to use to read the filesif I don't use Apple TV.

Any help most appreciated!

BJBBJB


BJBBJB's picture
Last seen: 5 months 4 weeks ago
Joined: 10/12/2011 - 3:06pm
Plus Member

Sorry for the long post above....was trying to share what has worked for me along with a few questions.

In short, any suggestions on what additional format to export to out of AME for a non-apple environment to stream/watch on HDTV?

Thanks,
BJBBJB


roblewis56's picture
Last seen: 2 days 23 hours ago
Joined: 06/07/2009 - 11:59pm

When I finish a project I produce it as a MPEG4 file and set the frame rate to the full 60fps (not 59.94). This is the format I upload to YouTube. To show it on my flat screen TV I connect my computer to a HDMI port on the TV using the display port on my laptop and a display port to HDMI converter and double click on the file.

Robert


dawgcatcher's picture
Last seen: 1 month 3 weeks ago
Joined: 02/08/2012 - 4:26pm

Thanks for all the additional thoughts and suggestions. I really like the file indexing advice, and really, that's what I am looking at. Simply hours of getting this footage ripped into a couple different places in a couple different formats - one for eay viewing, and one for editing later. Great advice on burning Blueray data disks for safekeeping. Sounds like a good wintertime project. :)


SpectateSwamp's picture
Last seen: 8 months 3 weeks ago
Joined: 09/12/2010 - 5:23pm

I have over 2200 videos up on youtube and a total of over 10,000 video clips. MPEG all of them.

My custom video player can pick a random video and a random spot and start play for a preset number of seconds then the next...

I have nearly 500 trapline clips on YouTube. What fun that was.

By hitting enter enter the playback switches to slow motion and causes the video to play a 2nd time..

It's best to run from a USB stick or external drive. You don't want to wear out C: that could be serious

http://www.telusplanet.net/public/stonedan/search.exe is the program

The latest feature (almost complete) plays a mix of LARGE text and video segments. Randomly.

I added this feature to make it easier to show off the Faster Than Sight objects. The Sky isn't empty.

http://www.archive.org/details/demo85-pict7is the original video on archive.org

I'll provide the instructions and the time-line playback so you can see all 9 or 10 important segments of the video in slow motion.


SpectateSwamp's picture
Last seen: 8 months 3 weeks ago
Joined: 09/12/2010 - 5:23pm

Here is what you need to try...

Put a copy of the exe on a USB stick or external drive and name it catmydrive.exe

That will catalog all the mpg videos you have placed there.

Rename another copy random.exe and that will play random videos from that drive.

Rename another copy randomseg.exe and that will play random video segments.

By hitting enter enter the videos switch to slow motion and back with another enter enter

That also causes the active video to play a 2nd time...

Download a copy of msvbvm50.dll here:

http://www.dlldump.com/download-dll-files.php/dllfiles/M/msvbvm50.dll/download.html

I include the above file with DVD's that I burn for others.

Run catmydrivejpg.exe to build a catalog for jpg and for mp3 catmydrivemp3.exe the catalog is in

mpeg_videos.txt for these so one may over-write the other

To see how it works for golf mpg video. run catmydrivegolf.exe

For golf we shut off recording right after the swing. This "golf" version plays the 10 to 15 second clip, then without re-opening the file it starts 6 seconds from the end. At 3 seconds left (the swing) it switches to slow motion. Then the last 3 seconds plays twice at full speed. Then the next....

Check the mpeg_videos.txt file to see how the time-line playback is controlled..


SpectateSwamp's picture
Last seen: 8 months 3 weeks ago
Joined: 09/12/2010 - 5:23pm

Want to see some Faster Than Sight action in slow motion?

Here is how...

http://www.archive.org/details/demo85-pict7 Download this 101MB file to a USB stick (it takes a few minutes)

place this small text file on the same USB stick.

http://www.telusplanet.net/public/stonedan/mpeg_videosmix.txt

take a copy of this program and place it there as well.

http://www.telusplanet.net/public/stonedan/search.exe

rename the search.exe to mpeg_videosmix.exe

And click on it

I'll be uploading more video playback for other archive FTS objects.

Cloaked mother ship is intercepted by one of the Swarm objects.

Spooked sparrow. He fluffs up his feathers an instant before the approach by a tiny object.

Super slow motion replay (multiple times) is needed to see these things.

Be prepared to get excited....


BJBBJB's picture
Last seen: 5 months 4 weeks ago
Joined: 10/12/2011 - 3:06pm
Plus Member

SInce I finally have a solution to my post above, It thought I would share it....

 

I setup a NAS RAID (networked attached storage RAID) device on my network.  A readynas but I am sure other brands would work as well. I have it in "mirrored" mode so my data is protected although I also back it up to another drive. It is fairly easy to set this up to "share" just like a folder on another PC.

 

I added a reasonably priced "streamer" to my network, a Western Digital Live Streaming media streamer.  Now of course many use these for online streaming etc, but I also use it to access my self-produced media...both raw and in DVD and Blu-ray form from the NAS.

 

I copied over all if my SD-DVD ISO files for completed projects. The player will actually play SD DVD menus directly from the ISO over the network just as if you were playing the DVD.  For Blu-rays, it works better if you convert the Blu-Ray to an .MKV file as it won't display a Blu-Ray menu so I backup the ISO files and then convert to .MKV files which is what I view.

 

Finally, for my raw footage that is not yet produced, I put it on the NAS with a descriptive filename, sorted in folders, and have direct access to this SD or HDV footage whenever I want.  I am sure at some point I will convert some of them to nicely produced Blu-Ray's or DVD's, but in the interim I at least have access to the video.

 

I find the above workflow provides more immediate access to my video media similar to my photo image media vs. waiting until a disc is produced.  Also, I have my raw footage captured and easy to access when I am ready to work on editing it.

 

I just kept this on the old thread because it was so on-point.  Hope that is okay.

 

BJB

 

 

 


dawgcatcher's picture
Last seen: 1 month 3 weeks ago
Joined: 02/08/2012 - 4:26pm

BJBBJB...Great post!  I know it was from quite a while back, but I am still restling with this and I am going to try your recepie.  Many thanks for taking the time to post this. Jack


BJBBJB's picture
Last seen: 5 months 4 weeks ago
Joined: 10/12/2011 - 3:06pm
Plus Member

dawgcatcher,

 

Glad it helped.  It is still working for me and hope it works for you.  I continue to post video much more quickly then I ever used to.  However I still wish Encore was appropriately upgraded to the rest of the "cloud program" standards as I find myself authoring fewer Blu-Rays which is sad.  So watching a lot more raw footage...

 

The only other thing I added is that I have found AVplayerHD app allows me to play the videos from the RAID on an Ipad (shudder)....not exactly an HDTV but convenient.  And even though it is a mirrored RAID, I make a static backup every 3 or 4 months to an external drive "just in case" there is a problem with it.

 

Good luck!

BJBBJB

 


John Sachanda's picture
Last seen: 3 months 2 weeks ago
Joined: 06/10/2008 - 2:32am
Plus Member Moderator

Wow!, what a great string of info. I am happy that I belong to such a great community of people willing to share their knowledge.

 

I don't have much to add other than a file naming format. I save a lot of files for the business and need to be ableto find things quickly. In orde to d this I set up main folders with a regular category naming format, i.e. (in your case) "My stuff", "My Family Stuff", then I set up subfolders i.e. "My Stuff/Hunting. Then I name specific files with a date and name format so I quickly can scan the order in which they came in and I start with the year as this puts the most recent on top, i.e. "14-0227  Today", "14-0226 Yesterday", "14-0225 Day before yesterday" etc. You can go on and on. I use windows, so you have to be aware of the filenaming limit as there is a specfic character count limit. I believe its 60 characters. You can always use the search feature to locate files quickly by date or key word.