Video Formats Explained

A few decades back, when Videomaker was just starting out, shooting and editing got a little complicated with the introduction of VHS, VHS-C and Betamax. If you read a two page article, you would discover the differences. When the digital age dawned, everything went out the window. Suddenly there was a bewildering array of video formats — .wmv, .asf, .rm, .mov, .mpeg. On top of that, many of these standards had their own substandards (MPEG-1, MPEG-2, etc.) How’s anyone supposed to keep it straight?

Containers and codecs

Possibly one of the most confusing things about video formats is the idea that there’s a “container” and “codec“. It’s enough to make you yearn for the days when you just put a tape in the camera and pressed record. The plethora of video formats means that, whatever type of video production you’re doing, there’s a good way to make it happen. Twenty years ago, everybody was watching movies the same way — either on a screen via a projector, or on a television set. Today many, many more options exist and people are taking advantage of them all. From high-end 4K home theaters, to video streaming from a cellphone, video is everywhere. By understanding the various formats, you can ensure that your video is viewed in the right way with the best quality.

An analogy of video formats

Figuring out exactly what containers and codecs are can be a little bewildering, because it’s a very technical subject. Think of containers as a type of publication. It might be a hardback book, a glossy magazine, a newspaper, a pamphlet or a gum wrapper. These all contain words, and potentially photographs, or images. Yet they each work in a different way.

Think of the video format like the way you view text or images in a publication. You could print Tolstoy’s “War and Peace“, for example, on Dove candy wrappers. It will take thousands of wrappers and who would want to read it that way? In the same way, you can create your vacation footage in an uncompressed format, but the file becomes enormous. There is no way you could upload it to the web or send it in an email. Similarly, you want your copy of “War and Peace” bound beautifully in a hardback book. If you are printing a takeout menu, on the other hand, you’ll want color photos on heavier paper. Words with images appear in a comic book, or a hardback book, or a newspaper. The images of a fashion magazine, however, require heavy-weight glossy paper to reproduce properly.

Every video application has a proper codec and container. Unfortunately, codecs and containers get improvements and updates regularly. You may rarely see today, what was a popular format a few years ago. Additionally, some containers and some codecs are proprietary. That means, occasionally, there are licensing issues related one type of container with another type of codec.

Data loss

Keep in mind, when you compress video data, there is some data lost in the process. Video compression applications work by looking for redundancies in a frame and maintain them frame to frame. For example, one bit of blue sky is the same as another bit of blue sky. The blue section carries though each frame. At high compression rates, this becomes obvious. At lower rates it’s difficult to notice.

Every video maker’s temptation is to use lossless formats to preserve all the original data. That isn’t practical for uploading, sending or storing your file. It is best to create multiple versions of your files for multiple uses. One file format you will uploaded to your web site. You may choose a different format or size to email to your clients. You might save the final product in a third video format to a hard drive for projection at an event.

Formats with lossy vs lossless compression

Image of woman with close up inset showing artifacts.

Edit and distribute in the highest quality

The highest quality video format is the one you originally captured. While digital files do not degrade in quality, every time they are converted, there is a lose in data. Converting your uncompressed files directly from your camera, even into a high quality file, does result in some loss of quality. It’s necessary to compress files in order to be able to share them. Avoid re-compressing any more than you have to.

Keep your master files in the original video format. Generally, you should edit and create versions at whatever sizes necessary. Whenever possible, you don’t want to convert a file from a file previously compressed.

You can rely on your editing applcation for lot of the work in deciding how to compress video files. Most consumer editing software today will have presets for various methods of distribution — such as email, YouTube or video display.

Editing software like FinalCut will gives you lots of choices depending on ho you want to use your video

Containers

Let’s define a few specific video formats and the different containers. A video’s file extension usually refers to the container. A few containers have codecs that they almost always use and other containers are often used with many different codecs.

Audio Video Interleave (.avi)

Microsoft developed and released this with Windows 3.1 way back. AVI files were once a workhorse of digital video. If I say “AVI is dead” the comments section will clog with people still using it. I’ll say that it’s popularity has waned, but you will find lots of legacy AVI files all over the web. Short answer, don’t output video to it, but keep a player handy.

Advanced Systems Format (.asf)

ASF is another proprietary Microsoft container. It usually houses files compressed with Microsoft’s WMV codec. To make things confusing, the files are usually designated .wmv and not .asf. The ASF container has the advantage over other formats in that it’s able to include Digital Rights Management. (DRM – a form of copy protection) Microsoft designed this format for streaming video from media servers or over the Internet. Short answer, again, don’t output video to it, but keep a player handy.

QuickTime (.mov or .qt)

Apple developed QuickTime and it supports a wide variety of codecs. It’s a proprietary format though and Apple decides what it supports. Like Microsoft’s version, .avi, Quicktime looked like it was going to fade into the sunset. It looked like it was about to as it die when Apple released the Maverick update. Quietly the update replaced anything inside a .mov container with h.264. In fact, both Nikon and Canon DSLR’s output h.264 video wrapped in a .mov container. Short answer: Sure, why not. Most people will be able to read .mov files for a while now.

Advanced Video Coding, High Definition (AVCHD)

AVCHD is a very popular container for data compressed with h.264. It comes to us through a collaboration between Sony and Panasonic as a format for digital camcorders. It’s a file based video format. That means it’s meant to be stored and played back on storage devices like flash drives or SD cards. This format supports both standard definition and a variety of high definition variants.

This format is an extraordinarily robust container. It includes, not just things like subtitles, but menu navigation and slideshows with audio. Short answer: Yes.

MPEG-4

MPEG-4 which was based on the Quicktime file format comes in a variety of extension including mp4, m4a, m4v, f4v, f4a, m4b, m4r, and f4b. Apple created this standard as an extension of MPEG-4. Originally the format was proprietary to Apple DRM to keep their files from playing on non-apple devices. You would use it, among other places, when distributing content on iTunes. The formats are so similar. In some instances, where the DRM isn’t being used, simply change the file extension to .mp4 and you can play the file.

There are many Codecs to choose from as you can see in this dropdown menu from Final Cut

Codecs

There are a number of Codecs out here and we have a good tutorial here. Two of the most common are:

Windows Media Video (.wmv)

Once the Internet became a primary delivery vehicle for things like video, people started trying to come up with ways to share video that wouldn’t take up a lot of bandwidth and disk space. One of the big advances was the idea of streaming video — where your computer downloads only a part of a video and begins to play while the download continues — this means you don’t have to wait two hours for a movie to download before you can start watching. Over the years the WMV format has grown to include support for high definition 720 and 1080 video. To make things complicated, files that end in .wmv are usually stored in an .asf container.

h.264

Not only do you need to call the MPEG-2 compression codec h.262, you have to keep from confusing it with the format used to compress Blu-ray disks as well as lots of web video. One of the very nice things about h.264 is that you can use it at very low and very high bitrates. The h.264 will send highly compressed low resolution video across the web and then happily encode your high definition movie at super high bitrates for delivery to a High Definition television. This is a very common codec for camcorders and digital video cameras. Its container is AVCHD.

What’s the best video format?

While there isn’t one “best video format,” there are best video formats for particular jobs. Ask yourself a few questions about your intended audience: Will they be watching video streaming over the Internet? What kind of connection speeds? Do they have a DVD player or a Blu-ray player? — are the longevity of the format and how widespread its adoption.

For a number of years, a good bet for a forward-looking, high-quality, versatile video format is h.264. (MPEG-4/AVC – Advanced Video Coding). H.264 is supported by a number of important players including Microsoft, Apple and Adobe. Though in early 2011 Google dropped support for h.264 from its Chrome browser. They cited the desire to use only open-source (i.e. non-patented, royalty-free) standards. Microsoft swiftly made a Chrome extension which restored support. Google’s answer was rolling out .vp9 (later vp10) used as part of HTML5. Performance of the two codecs is very similar. Adoptions and usage will determine if there’s an eventual winner.

Switching between video formats

If you do transfer between formats, remember that re-compression causes degradation. If quality is of paramount importance, don’t delete the originals, archive them somewhere. Be sure to note that when moving between containers, data streams like subtitles and chapter data may be lost, if the new container doesn’t support them.

Conclusion

As viewing experiences and platforms evolve, video delivery will continue to evolve and change. You will see new codecs and containers that will deliver larger amounts of data more quickly and with additional data streams. There will never be a final format. Just keep your audience in mind and the type of video you want to deliver. There will always be some options that are superior to others. Good organizational skills and regular migrating to new formats will make sure that your video survives as technology changes.

18 COMMENTS

  1. This is a great article ,very informative, thank you. We are setting up live stream for our church. We actually switched from windows to mac. I would like to know which camcorder or professional camera under 1000 dollars that can do the job. I’m using wire cast as our encoding software and we’re using a Mac with 10.8

    I have more info. I can give you, but didnt want to inundate in a post. Thank you!

  2. Hi, firstly…great article. Very simply put. 

     

    I have an old .avi and only need to play it on a large TV in reception. Its only 800MB so I thought I would avoid compressing it altogether in order to keep the 'quality' (some of the video is over 10 years old). 

     

    However, it does not play on any media device I have got but will play on my PC. The biggest compressed version i have got is a .vob file and it works fine. 

     

    Any ideas why it would not work? As you say above it only needs to be compressed to share and the media device holds much more than 800MB so whats the issue?

     

    Clearly my PC doesnt think there is an issue. 

  3. It is also worthy of noting that not all stand-alone devices (DVD players, television sets etc.) support all codecs/containers, and not equally.

     

    What is important to remember – and this may answer Karl Russell's question – is that, on a computer (be it a Windows/*nix PC, or a Mac) appropriate codecs can be installed as required, and processing speed is a non-issue! As in – even if the video playback is choppy beyond reason, this is still the user's problem. It's up to them to lower the quality (or find a lower-quality file), or upgrade their machine.

     

    But! Not so with stand-alone players! These devices usually can't be upgraded, and certainly their hardware can NOT be refreshed, except by buying an entirely new unit. They don't have the kind of structure a PC does. Instead, all decoding is done in specialized chips that do nothing else. And as it is, the more calculation-intense an operation is, the more advanced a chip it requires – and the more pricey it becomes. In the interest of keeping design simple and prices competetive, corners are cut. (find me a stand-alone player, whatever it is, under $350, that can play High or even Main h.264 video with all its bells and whistles, I dare You) Enough to say that way back in 2006 You would be hard-pressed to find a device that supports even a hint of h.264 – it was too intensive to process back then, and everything was DVD really (h.262).

     

    As such, support for many options within the codecs is usually dropped. Many older codecs are also dropped (to simplify things). It's a closed environment that nobody except the maker can modify. And even though the companies try hard to deliver robust devices, there will always be gaps – by necessity.

     

    And even today, even with the widespread adoption of h.264 as a standard — h.264 still has many options that have not been explored by device manufacturers. Why? Simple – they provide little gain in image quality, and require a lot of processing power in exchange. Of course, these little tricks that h.264 can do are the little details that set the mark between "great" and "awesome" video quality.

     

     

    But what I'm trying to get at, the most, is the fact that especially when delivering to stand-alone playback devices, it is of utmost importance to first and foremost keep the final device in mind when compressing, and compress for it. Finding out the exact specs can be an issue (esp. with the cheaper models) — manufacturers all tend to boast support for many codecs, but when they're up against the wall about exactly which options and variations they support, the fall silent for some reason. A file encoded in h.264 that plays on one device may or MAY NOT play on another. And that's where the problem is at.

  4. Some good info here, but definitely needs to be updated.  Claiming that Adobe will support Flash for some time, no mention of h.265, and only an external reference to html5 standards are all glaring errors from the last year of technical developments. Since this article is promoted on the Videomake site, it would serve us readers well to post some updates, please.

    • honestly, yeah there have been developments, but they have only been accepted by a small minority of places. h.265 for example is not supported by most browsers, and not even any major browsers. I don’t think this is aged as much as it is targeted toward a wide, shallow audience

  5. H.264 
    Not only do you need to call the MPEG-2 compression codec H.262, you have to keep from confusing it with H.264, which is used to compress Blu-ray disks as well as lots of web video. One of the very nice things about H.264 is that you can use it at very low and very high bitrates. The H.264 will send highly compressed low resolution video across the web and then happily encode your high definition movie at super high bitrates for delivery to a High Definition television. This is a very common codec for camcorders and digital video cameras. Its container is [b]AVCHD[/b].

     

    It can be, but it is not limited to that…

  6. I recently had a vhs cleaned up.  I received the file back in huffyuv codec (22GB).   I installed the the codec in Windows/Syswow64 as directed.  I was able to play it back on XMedia recode and was very impressed with the color and clarity.  However, I was unable to use it on Premiere or Pinnacle 18 editing systems.  I finally recoded it in XMedia Recode to FFV1, altho it came out a little smaller, 11GB.  Then I was able to edit it in  Pinnacle.  Is there anyway to get Premiere of Pinnacle to "see" huffyuv? 

  7. I want to convert my father’s 8mm home movies to a digital format with an editable master file. I’m a total editing novice, so I think a PC-editable format might be good enough. (But I’m thinking of transferring using a Pro 2K process.) An online company (videoconversionexperts.com) outputs to HD-AVI, and they don’t offer AVDHD. They do offer MP-4 but call it streaming quality rather than editing quality. Your column says AVI is a dinosaur, and you wrote that four years ago. Am I missing something here?

  8. VIDEOMAKER PLEASE BRING BACK NEWLINES FFS >.< !!!! | | | AVI is indeed a dinosaur. For one, it doesn't support timestamps on frames, which makes it pretty bad for NLE. | arnold68: the company is bullshitting You. There is no such thing as "HDAVI" or "HD-AVI". | AVI, MP4, MKV and OGM -- these are all containers, which means they can contain individual streams compressed with CODECS. MP4 typically holds H.264 video (this is the video stream encoded in H.264). AVI however does not support independent streams, instead opting to interleave video and audio together (presumably for the fact that it was less demanding and simpler to process back when it was introduced in 1996). As such, they are practically never used for anything nowadays, unless it needs to be simple and cheap. Even in-game cinematics have shifted from AVI to more controllable things like OGM or even RADVideo (which is a story unto itself). AVI spec is 20 years old this year - it's ancient by all accounts. | | More notably, AVI was never meant for HD work, even though theoretically you can save such a stream to AVI (because it's essentially a container). It will probably violate AVI format specs (?), but it's doable. | | Second, it's not AVDHD - it's AVCHD (AVDHD is appartently a DVD player by Audiovox, or somesuch). If You want to edit the file, You will want edit-quality, and that'll be something like MPEG HD422 (in a container like MXF or MP4) or PRORES (in a MOV container), or something similar. | | In general, the point being -- there are two kinds of codecs/formats out there: edit-oriented and playback-oriented (streaming, also). | --- Playback-oriented ones will ensure good quality for as little processing power required and/or as little storage space required (ex. h.262 (MPEG-2 DVD), h.264 (MPEG-4 BluRay)). They achieve that by encoding data based on relations, so that the next frame may be "previous frame + changes" instead of an entire new image. A few may be "visually lossless", but some information (usually quite a lot of it) is typically lost regardless. The advantages are: +small size, +easy decoding, +low bitrates. | --- Edit-oriented ones will ensure each frame is kept separately (exclusively I-frames) instead of relying on predictions and changesets/relations. The resultant files are positively HUGE, but should ensure that the result coming out of an NLE (Non-Linear Editor, like Premiere Pro or Final Cut Pro) will be of equal quality (they tend to be lossless codecs). Of course - handling 25+ frames per second at full-frame sizes is taxing for the storage device, so real-time playback is rarely feasible with these (hence they're useless for playback/streaming). | | A normal editing process typically sees editing on full-frame codecs, followed by compression to streaming formats (as required) for final delivery. | From where I'm standing, they're probably offering You a mixed bag of something they think will be editable, but may not be. I would stay away from those folk, or at the very least request they provide their original pulled material, even if it's all TIFF or DNG frames (which will be reasonably HUGE!). | | If You want to edit on PC, Your editing go-to program will probably be Premiere Pro, and Premiere Pro natively works with AVCHD and DNxHD at high bitrates. But do not worry, You can often use an interim tool (I like to use ffmpeg or ffmbc, there are nice GUIs for using them through windows -- look up "AnotherGUI") to convert whatever the company gives You to Your prefered editing format.

  9. It seems that working with huffYUV is broken on many systems, for whatever reason. Recoding it to the open standard FFV1 is the right choice here (I would’ve probably gone with ProRes 4:2:2 HQ, but I’m typically on a Mac).
    |
    |
    On a side note, Premiere Pro seems to be rather picky about its input video formats, so I wouldn’t rely on it as the industry benchmark… 😉

  10. Nooooooooooooo! You pronounce “da-tuh” and “day-tuh” for the word “data” in your video. Please choose one version and stick with it. I humbly suggest “day-tuh” because we all know that’s the correct pronunciation. Thank you and good day to you kind sirs.

  11. I am trying to get a homemade (Premier Elements) travel video saved as H.264 to play on a Panasonic Recorder: Smart Network 3D Blue-ray Disc/DVD Player HDD Recorder which will only accept AVCHD/XVID/MKV As a novice, I am at a loss to know which video codec will play. HELP, please?

  12. I use video editing software every day such as: Premierer, Camsta, AE …Without knowing their formats. Now i see.
    The article is very detailed and complete, all aspects of the video are included in this.
    Thanks.

LEAVE A REPLY

Please enter your comment!
Please enter your name here