Advanced Video Compression - Part 2
Various Codecs, Formats and their Pros and Cons

MPEG1 - The Golden Oldie

Origins: Back in the very late 80's, the Motion Picture Experts Group was trying to come up with an open, efficient video compression standard which was officially brought into the International Standards Organization as ISO/IEC-11172, or more commonly known as MPEG1. The MPEG1 standard was designed originally for 1.5MBit/second datarates (i.e. 1X CD-ROM speed, the 1.5 figure includes audio however) and 352x240 resolution.

How it Works: MPEG relies heavily on Inter-frame compression. There are 3 different types of frames in the MPEG standard - Intraframes (I-frames), Predicate frames (P-frames), and Bidirectially-interpolated frames (B-frames). I-frames are essentially the MPEG equivalent of Keyframes - these frames have all the data necessary to recreate themselves, i.e. you don't need to reference another frame to render an I-frame. P-frames reference the previous I or P frame and store the changes in picture. B-frames are meant to be very low-bitrate frames, so they reference both past and future frames. The standard MPEG stream structure used in VCDs, SVCDs, and DVDs is IBBPBBPBBPBBPBBPBB, although the official MPEG standard doesn't place any restrictions on the distance between I or P frames. A group of frames that starts with an I frame and ends at the frame before the next I frame is called a Group of Pictures, or GOP. It's perfectly legal to stop a GOP short, insert a new keyframe and begin a new GOP at any time. This is akin to Variable Keyframe Interval in other codecs.

Benefits: MPEG1 was the codec which made distributable digital video happen. MPEG could be streamed, stored on CDs - you could do all sorts of stuff with it. Even today it is still a very viable compression algorithm. You can play MPEG movies on almost anything, from settop players, to PCs, to handhelds running PocketPC! That's pretty darn impressive!

Disadvantages: MPEG1 is old, lets make no bones about it. It's still damn good, and it beats out everything else in terms of compatibility, but in terms of technical quality there are better things out there. In order to get a good picture, it does require more bits/second than other codecs.

Recommendations: MPEG1, while it won't make the smallest files, is the KING in terms of compatibility. If you want everything under the sun to be able to play your AMVs, distribute in 352x240 MPEG1 files. Otherwise, I'd stay away from it.

MPEG2 - The Behemoth

Origins: After the MPEG1 standard was finalized, and people started trying to apply it to higher-resolution video pictures, there were many flaws in the standard that became apparent. The biggest of these was that MPEG1 could only compress progressive-scan images, which meant real TV pictures (which were interlaced) were very difficult to compress. MPEG2 was born out of the desire to achieve compression of broadcast-quality video, and to this end it has succeeded. MPEG2 is used for DVDs, ATSC (High Definition Television) broadcasts, Personal Video Recorders (such as TiVo), and many other applications. MPEG2 is so versatile that while originally it was planned to have an MPEG3 standard for High-Definition TV broadcasts, it turned out that MPEG2 scaled in terms of bitrate so that only 1 standard was necessary for both Standard and High Definition video.

How it Works: MPEG2 is very similar to MPEG1 when you look at the surface, although several of the underlying pieces have been completely changed. But for our purposes, it's good enough to say that MPEG2 is a souped-up version of MPEG1 with support for Interlaced video and High-res pictures.

Benefits: MPEG2 gives better picture quality than MPEG1 at full CCIR 601 resolution (720x480) and at comparable bitrate. Basically, MPEG2 scales, while MPEG1 doesn't.

Disadvantages: MPEG2 takes more CPU horsepower to decompress, and basically has no advantage over MPEG1 at lower resolutions. If you're distributing for the web, you're not going to want to be compressing stuff in MPEG2 instead of MPEG1 because the benefits of MPEG2 aren't realized until you get into full TV resolution, which means big filesizes.

Recommendation: If you're creating your own high-quality archival backups of your videos, I would encode them at DVD-quality MPEG2, but I would never use MPEG2 for online distribution.

ASF - Advanced Streaming Format, with the emphasis on STREAMING

Origins: Microsoft, seeing the success people like Apple and RealNetworks were having in the streaming video arena, they decided to barge in, ignore the previous standards and create their own, as Microsoft usually does. Thus ASF was born, a watered down version of AVI paired with an extra information stream. ASF is both a format and a codec. It's basically AVI that can only use Microsoft's MPEG4 implementation, but without the indexing issues of AVI (you must have all of an AVI to play it as index blocks and such are referenced at the end of the file).
A newer, standalone/streaming version of ASF called WMV was introduced not too long ago, but it's essentially the same thing, including the drawbacks of data overhead and such.

How it Works: Essentially, ASF is Microsoft's own partial implementation of the MPEG4 standard. I say partial because if you read the MPEG4 standard, you will notice that there's a hell of a lot more to the standard than what Microsoft provides. Basically they just took the video encoding portion, stripped out things like B-frames which weren't compatible with the AVI format, and wrote an encoder/decoder for it and called it Windows Media.

Benefits: If you want to stream your video over the web, ASF may be very attractive to you. It's a format which was meant for streaming, so most of its architecture is centered around that fact. However, I doubt most of you have the bandwidth to stream your videos live over the web.

Disadvantages: Just as it's an advantage for streaming, the structure of ASF is also very prohibitive when you're trying to simply store video in single files, to be downloaded and viewed later. ASF keeps track of all sorts of other things in its stream, meaning there's significant bitrate overhead for non-video and non-audio data (example, using the old VirtualDub 1.3c and the 4.0 beta of Windows Media Tools, where the MP43 codec still worked in AVIs, an average ASF file takes up 25% more space than a direct stream copy of the video stream to an AVI file). ASF is limited to 352x288 resolution. ASF is also prone to errors, as when you compress to ASF the encoder does things like dropping frames to maintain bitrate, and it also does some other nasty things which when streaming isn't an issue, but definitely is when you're just trying to encode standalone video files.
As if that all weren't bad enough, Microsoft is also very protective of its format. They've patented the ASF file structure so that no program can interface with it without breaking patent law, and their license agreements (or at least they used to) actually forbid transcoding to other formats after encoding to ASF.

Recommendation: Don't use it unless you're planning on streaming your files.

DivX ;-) 3.11a Alpha - ASF without the drawbacks

Origins: A guy who went by the handle "Gej" used a hex editor and some other tools to modify the binary dll file of Microsoft's MS-MPEG4v3 ASF codec so that it would work under AVIs, so that you could use it to encode high-res movies (since ASF was restricted to 352x288 resolution). He then released it onto the net. This became the original DivX which was updated until version 3.11alpha.

How it Works: Essentially it's the same codec as what's used in ASF (if you don't believe me, use the ASF VirtualDub to extract an ASF video stream to an AVI file, then change the AVI's FourCC code to DIV3 and notice it still works) except that it uses the AVI file format, meaning no resolution restriction, no filesize overhead, and easy manipulation and playback in any program supporting AVI.

Benefits: DivX, like ASF, is a decent MPEG4 implementation (although incomplete). It can compress images better than MPEG1 in less bits, but it does lose more color data than MPEG1 (at least in Microsoft's implementation it does). However it's since been surpassed in quality by XviD but still has better compatibility.

Disadvantages: DivX isn't quite as compatible as MPEG1 - it plays on PCs, BeOS, and Linux easily enough, and it will play on the Mac with some work, but on other platforms you're out of luck. It also takes more horsepower to playback than MPEG1 (not as much as MPEG2 though), but there are settings in the advanced area of the Codec for playback on slower machines at slightly lower video quality. DivX also has some issues with solid black not being black.

Recommendation: At this time I would recommend encoding to XviD if you want MPEG4 encoding, but DivX3 is still a very viable codec and when used properly with NanDub it can make very good encodes.

Origins: After the success of the original DivX3 codec, a group got together and started an open source version of DivX from scratch called OpenDivX. Unfortunately, they later closed the source and started working on it solely in house. First they released DivX4, which was alright but wasn't as good as DivX3. With the release of DivX5, they managed to hack in B-frames into an AVI file, which is a pretty impressive feat, although they way they do it is very messy and produces AVI files with rather bizarre structure, but they'll playback via DirectShow just fine.

How it Works: DivX5 is a pretty standard MPEG4 codec, and in fact can produce MPEG4 compliant streams if you encode audio to AAC and distribute as .mp4 files. However you'll need an MPEG4 software player or wait until Microsoft includes MPEG4 playback in their media player.

Benefits: DivX5 is pretty easy to setup and work with, but it doesn't have very many other advantages for encoding. It's playback filter, though, is fast and offers good post-processing capability. The playback filter's also easy to install and works good on properly encoded XviD files so we actually recommend installing DivX5 even if you don't plan on encoding with it. DivX5 does have very good playback options for PC, Linux, and the Mac, which covers the three biggest desktop platforms.

Disadvantages: DivX5's quality, while pretty good, still doesn't surpass the original DivX3 in my opinion, and the inability to really tweak the codec's internal options are a big downside. Also, while its B-frames do improve compressibility, they look very bad and so I wouldn't use them anyway. Also, in order to get all the options you have to either pay money for it, or download a version with adware and spyware attached to it. While it's possible to remove it, I think it's insulting and I don't want to get it near my machine. Fortunately you can download a full-fledged decoder in the non-adware version.

Recommendation: I don't recommend encoding your movies in DivX5 due to its questionable quality and adware. DivX3 as well as XviD are better options for MPEG4 encoding, IMO

Origins: When OpenDivX closed its source and went on to produce DivX4, a group of open source developers decided to take the code that was left from OpenDivX and start their own codec. Thus XviD (or DivX backwards) was born. By now very little of the original OpenDivX code remains but there's a few bits here and there.

How it Works: XviD is an MPEG4 compliant codec which performs very well and has oodles of options. While it doesn't do B-frames yet (and when they do they may not be compatible with the DivX5 playback filter) it does implement many of the options of MPEG4 and very well, at that.

Benefits: XviD has all sorts of tweakable options, which means an experienced encoder (or one following a well-written guide) can produce some very nice looking encodes. It can also produce MPEG4 compatible streams if you want to distribute as .MP4 files. Usually the latest snapshots released by Koepi are pretty stable and a good representation of the state of the code, and since XviD's always producing MPEG4 compatible streams, you should never run into a situation where your old encodes are no longer compatible. The new versions will just do some things better or add new features. Also utilizing some of the programs provided by DivX Networks, you can get XviD files to playback on the same platforms DivX can, which includes PC, Linux, and Mac.

Disadvantages: Unlike DivX5, it can be difficult to setup and get something that looks good. It also is still in its alpha stages of development, which means there may be bugs. XviD also lacks it's own good DirectShow filter. The XviD specific filter has very few good post-processing capabilities, and while ffdshow will do decent decoding, it's not very user-friendly and changes too often to be a good thing to recommend your viewers install. Fortunately you can encode your XviD files to playback correctly with the DivX5 playback filter.

Recommendation: This is what I use exclusively now to distribute my files. It provides excellent quality and compressibility as well as advanced features, and it has good cross-compatability amongst desktop operating systems.

Quicktime - Apple's video platform

No, Quicktime isn't a codec in and of itself - it's more of a format like AVI is. However, I recommend against distributing your videos as any Quicktime format because of the requirements of its player. While not as intrusive as say RealPlayer, Quicktime's player for Windows is slow, clunky, and unstable. Also QT playback in Linux is sketchy at best. While Apple claims to have a fully compliant MPEG4 codec in their Quicktime Version 6 package, tests have shown that it does not play back real MPEG4 streams, nor will compliant players playback it's MPEG4 streams. Consider QT's MPEG4 codec to be the same as Microsoft's MPEG4 codec - they're open standards, yet proprietary in practice.

ErMaC - September 2002