Video Compression

What is video compression?

Raw uncompressed digital video is HUGE. A single frame of video at a resolution of 720x480 is about 1MB in size. This means that a single SECOND of digital video will use up 24-30MB of space! If we start talking HD resolutions, then these figures would skyrocket. In order to make digital video actually usable, some method of reducing the size had to be invented.

Probably the earliest, and truly successful video codecs (CODEC stands for COmpressor/DECompressor or COder/DECoder) was MPEG1. Nowadays we have the much more advanced format, MPEG4, which compresses even better.

But how does compression work? Well, like with any kind of compression, there are two kinds: lossless and lossy.

Lossless Compression

Lossless compression, as the name implies, means that after compressing the video, and then decompressing it, you wind up with the exact same data as you put in. This is comparable to something like ZIP or RAR (In fact, one of the more common lossless codecs out there, Huffyuv, basically uses the same compression algorithm as ZIP on each frame of video to achieve its compression). Lossless has the advantage that no matter how many times you compress it, you still haven't lost any video data. The bad part is that most often you don't save nearly as much space as you would with other lossy compression algorithms. There are many different lossless codecs these days, some of which are described later on in this guide.

Lossy Compression

This is the form of video compression most people are familiar with. The majority of all video codecs out there are lossy, meaning that when you compress the video and then decompress it, you do not get back what you put in. Now, this isn't as bad as it may sound. Obviously if you're compressing something like a text document, you don't want to lose any of the data, but with something like a picture, even if a few bits and pieces aren't quite right, you can still make out the general gist of the image.

But, because lossy compression throws some of your data away, we must be VERY careful about how we use it, because we can't get that original data back! To compound matters, each time you compress something with a lossy codec, more and more data is thrown away, making it look worse and worse! So ideally, we only want to compress with a lossy codec ONCE--your final distribution copy of your video.

Now with that distinction out of the way, we can discuss how to compress the actual video itself, of which there are also 2 basic ways of doing this. These two methods are called intra-frame and inter-frame compression.

Intra-Frame Compression

As the name suggests, Intra-frame compression relies on only the single, specific frame it is working with to compress the data within it. This means that you are basically encoding each separate frame as its own picture. In the case of something like MJPEG (an algorithm which uses only intra-frame compression) you are encoding each and every frame using a JPEG compressor (which should be familiar to anyone who has ever worked with images, it has the extension jpg). This means that while you can't take advantage of the information in previous and forthcoming frames, you have the ability to recreate each frame without the need for the others. This comes in very handy when editing video (which is why editing with a pure intra-frame codec is a must).

Inter-Frame Compression

As should be obvious by now, Inter-frame compression relies on information in preceding and occasionally forthcoming frames to compress an image. The most well-known way of exploiting this data is by exploiting the fact that the majority of a video image isn't always moving. Take a newscast, for instance. Usually the only thing moving is the anchor's body, while the rest of the set is staying perfectly still. Why should we bother to store all of the data that makes up that background for every single frame? Here's an illustration:

We start with this frame. Now we take a look at what is actually changing in between frames:

Here are frames 2 and 3, with the only parts that have changed since the previous frame shown below the actual frame.

Here we have frames 4 and 5 with the same arrangement.

Notice that the only things in the frame that changed was the chair moving (and redrawing the background in the area that the chair had left since the previous frame). Almost all inter-frame compression is based on exploiting this fact about a video image. The disadvantage to this is that if you want to check out frame 5, you can't actually see what the real frame looks like without first looking back at frame 1, then applying the changes in frames 2, 3, 4, and 5. This leads to the notion of keyframes. A keyframe is a special frame which exists on its own - in other words it doesn't rely on any other frames to store its data. Thus seeking to a particular frame usually involves first going to the preceding keyframe and then applying each successive partial-frame until we reach the desired one.

This presents a problem for video editing - if we want to cut the video on a specific frame, most programs have a very good chance of getting rather get confused unless cut at a keyframe. What I mean by this is, if you want to cut at frame 3, and that isn't a keyframe, sometimes your editing program might make the cut right there at frame three... but sometimes it might actually cut at frame 2, or frame 4! This makes your editing options very limited unless you create lots and lots of keyframes, in which case you then lose the benefits of inter-frame compression in the first place!

Bottom line: Use Intra-frame compression for editing, and inter-frame compression for final distribution and archival.

August 2010