Encoding DivX 5-Compatible video with XviD 1.0.1+

Before you Encode

1) The footage is NOT interlaced.  looks terrible on monitors and causes havoc for encoders. You must remove interlacing before encoding - see this page on deinterlacing.

2) Your footage has been cropped. Having source footage with black borders on any side of the video is a waste of space that could be used to encode the actual active image. These should be trimmed no matter how small.

3) The video is resized to multiples of 16. Your video should be resized to a resolution that fits the source aspect ratio (usually 4:3). It must be resized to multiples of 16 in order to be fully compatible - 320x240, 512x384, 640x480 are example of 4:3 resolutions that are multiples of 16.

4) Your video has been temporally smoothed. Pretty much every source has some kind of noise that happens between consecutive frames - the only time this wouldn't be true is if you have drawn the footage cleanly yourself. Even the prettiest DVD has this noise and it wastes a large amount of data when encoding. Temporal smoothers such as Temporal Cleaner, Convolution3D, deen, fluxsmooth and so on all reduce this and give you a lot more bits to play with when encoding the parts you really want to look good.

1. What is this XviD thing?
2. Where to get XviD
3. Getting Started
4. VirtualDubMod
5. XviD 1.0 Compression Options - What's all this about?
6. Setting up XviD 1.0 Encoding Options
7. Twopass Encoding
8. The First Pass
9. Saving the First Pass
10. The Second Pass

What is this XviD thing?

XviD is an mpeg-4 codec which is developed for educational use by programmers in their free time. The reason you are reading an XviD guide here instead of a DivX5 guide, WMV9 or RealPlayer guide etc. is as follows:

1) XviD is more customisable, which allows us to choose options better suited for amvs - this is not entirely possible with DivX5.
2) Although RealPlayer 9 and above can give very good results, it is a restrictive format that doesn't allow for as much compatibility as MPEG4 variants do.
3) XviD doesn't have any spyware of any kind; DivX 5 (if you want all the options) installs spyware unless you pay for it.
4) I've got more Xvid experience :)

The other good thing about XviD is that although you have to download a new codec to encode your videos, people who have DivX 5 do not need to download a new codec to see them, provided you use the options in this guide :) Also, as it is mpeg4 compliant it can be later muxed into a true standards compatible MPEG4 file or played back on MPEG4 hardware players.

If you want to encode XviD avis with extra options, other people will need either the XviD codec or FFDShow installed to watch them.

Where to get XviD

An installer is contained in the amvapp, but you can download binaries from koepi's site. In addition the gamrdev site has even more up-to-date builds in terms of code but they are untested.

Getting Started

This guide is based upon the XviD 1.0.1. The version of XviD is included in the AMVapp and can be downloaded from Koepi's Site.

Make sure you do not use the 1.0 release, as there were a few bugs present that would affect anime encoding.

It is possible that by the time you read this, the 1.1 branch will be under way and betas will be released. It is likely that the newer betas will have better quality but I cannot guarantee that the options in this guide will work with the newer developments nor that the encodes you make with those betas will be compliant and work in the future. I will hopefully try and keep this guide as fresh and up to date as possible.

Note well: Before installing a new copy of XviD you should close down any video programs like Premiere, Virtualdub and so on and uninstall the old one first. This is particularly important if you have a very old version of XviD as you can have many problems if a new version is installed on top of the old one.

VirtualDubMod

When you have VirtualDubMod open, you can now go to File > Open Video File... to load your source footage. If you are following these guides correctly you should have an avisynth (.avs) file to load up which has your filtering and resizing options all ready and waiting. Now choose Video > Compression, scroll down and select Xvid MPEG-4 Encoder then click the button labelled "Configuration"

XviD 1.0 Compression Options - What's all this about?

If you are familiar with the old XviD encoder layout you will notice that things look a LOT different these days. You will be greeted with the following which will now be referred to as the main page.

Let's look at each part individually so you can work out what it all does and what parts you need to know about.

Profile @ Level: This pulldown menu allows you to select MPEG-4 encoding levels. These levels are often specified by decoder devices and define what sort of mpeg-4 content they can play. AS means Advanced Simple and L5 is level 5. AS @ L5 is the setting we will use in this guide. You can choose "unrestricted profile" but your encodes are not guaranteed to be MPEG-4 compatible. The more... button will take us to the settings related to Quantizers and so on - we will come back to this later.

Encoding Type: Like before, you can encode Single Pass, Twopass (First Pass), Twopass (Second Pass). There is no distinction between internal and external twopass operations now. Of course, you will want to select Twopass (1st Pass). The more... button next to this menu will contain different options depending on the type of encoding selected. Again, we will look at these later.

Setting up XviD 1.0.1 Encoding Options

Notes about Twopass Encoding

The First Pass

Interlaced Encoding is an option you will never need to use. If you have followed these guides then you should not have an interlaced video to compress, nor should you be compressing an interlaced video if you are going to distribute it on the Internet. This option encodes the footage in fields instead of frames to retain any interlacing. It does not remove interlacing, it is designed to keep it. This function is not always compatible with non-xvid players and is not recommended.

Quarter Pixel is a neat feature that isn't always helpful. It's designed to improve the accuracy of motion detection (it can calculate motion to 1/4 of a pixel in theoretical movement accuracy instead of just 1/2 like normal) and this can improve compression in some sources. However, storing information at this greater accuracy takes up more space straight away so you have a Catch 22 situation because it will only save bits if there is a lot of quarter pixel movement in the source. You cannot tell beforehand, it's all in the maths and so it will not help every encode which means you may end up with a bigger file with no benefit. It also takes much more processing power to decode (up to 60% more sometimes) so because of these issues it is not recommended for amv files for mass distribution.

Global Motion Compensation takes the idea of calculating motion and applying it to the whole image - this can save bits overall and is a good thing. The only 'problem' is that the XviD encoder is very sophisticated and uses 3 warp points to calculate this whereas the DivX 5 encoder only uses 1 (at the moment). Unfortunately, decoders aren't entirely up to speed with this either so it is common for them to not know how to properly handle GMC with 3 warp points. For compatibility reasons this option is not, at the moment, recommended.

B-VOPs are B-Frames, a feature that DivX 5 touted but in their early versions were pretty ugly. XviD's new B-Frames are pretty nice and very useful as I shall explain. You may recall from the theory guides that there are Intra-frames (I-Frames),  Predicate Frames (P Frames) and Bi-directially Interpolated Frames (B-Frames). The old XviD used to only use I and P frames. B-Frames are used to save even more space by referring to data both in the last frame and one that is coming later in a sequence. They are incredibly useful in saving space as they can be compressed more while the loss in quality is less than or equal to the loss in bits which means that overall you have more bits to use which improves quality overall and everyone's a winner. You don't have to use B-Frames (particularly if you aren't interested in saving space and just want the best quality) but they are good for getting the best quality in high compression. It is recommended to keep this option checked unless size is really not a problem for your encoding.

Max consecutive BVOPs specifies the maximum amount of these B-Frames that can appear in a row. DivX 5 only encodes 1 B-Frame in a row. XviD's default is 2 which is the highest you should go (as each extra B-Frame increases the processing power needed to decode). However, there is a compatibility concern. DivX5 can decode 2 B-Frames in a row but not when Packed bitstream is selected. This guide recommends keeping 2 B-Frames and deselecting Packed bitstream but if you want to keep things really compatible with DivX 5 and some hardware players then you may want to reduce the max BVOPs to 1.

Quantizer ratio defines how much higher the B-Frame quantizers will be than the P-frames. Higher quantizers are lower quality but the B-Frame savings make them worthwhile. The default setting of 1.50 is fine and recommended.

Quantizer offset: adds this value to the B-Frame quantizer number calculation. Keep this at the default of 1.00.

Packed bitstream is a function established by DivX 5.1 which in theory allows better decoding and reduces bad audio sync. This is true until you start using more than 1 B-Frame. If you use 2 B-Frames this should be deselected in order to ensure that people will be able to watch this encode correctly using the DivX 5 decoder.

Closed GOV makes sure that a P-Frame always appears before a new I-Frame. You don't have to understand why except to know that this is a good thing. Always keep this selected.

These are the main options that you will need to set when making anime encodes and will work perfectly fine for live action encodes too. These options will also make sure that your encode can be easily played by a variety of decoders including DivX 5 and ffdshow.

When you are confident that all of these options are set correctly, you may click OK to return to the main encoder options page.

You may be wondering what the other tabs were for. The Level tab gives you some stats on the MPEG-4 profile and level you have selected - these are theoretical values and won't have any effect on your encode. The Aspect Ratio tab is, presently, only useful if you are going to use a special container for the video (such as MPEG-4 or Matroska) as it allows you to set an option to tell a decoder to resize to a particular aspect ratio on decoding. This is designed to encode anamorphic footage in mpeg-4 containers. Your video should not use this option as it will be already resized to square pixels which is the most compatible method for decoding as the avi container is not really used for specifying aspect ratio and many players will ignore this setting.

Now that you are back on the main page, you should make sure that Twopass - 1st pass is selected in the Encoding Type pulldown menu. Doing this will hide the bitrate window as the First Pass of an encode does not attempt to reach a particular bitrate as it is used for calculating quality distribution in your video.

Click the more... button that is next to Encoding Type and you will be greeted with a simple menu with a file location window and two options.

Stats filename: The statistics that are recorded during your first pass are kept in a log file. You can, if you like, specify a particular file to save this to. By default it is saved in the directory of the file you are encoding, so if you encode something to C:\encode\amv.avi then the stats file will be C:\encode\video.pass - you don't need to change this option.

Full Quality First Pass: This is an option that almost wasn't included until myself and several others asked for it to be put in. By default, XviD does a quick 1st pass that doesn't really create a very stable encode - it's just done to make the calculations for the second pass. You can (and should) select that the first pass creates a file that is the full quality of the encoding settings. The reason is twofold: 1) The first pass may be small enough to distribute and so you know you have the best quality encode you can get with these options and this source. 2) It's useful to know how big the maximum quality is to help you calculate how small you could make your file without losing too much quality. You should select this option - it will be a little slower but it's much more useful. However, if you know that the first pass is definitely going to be too big, disabling full quality first pass will be much faster.

Discard First Pass: This sets XviD so that it doesn't really save any video data to the file during first pass. Again, we actually want to keep the first pass in case it is good enough. Uncheck this option.

You can now press OK to return to the main page.

We are almost done with our settings (I know there's a lot of them but you get used to it after an encode or two). You won't need to specify Zones - if you think you do then you probably already know what they are used for. 99% of the time you won't need them from amv encoding. The only thing I will mention is that in the Zone options you can set a Zone to use a Chroma Optimizer which is fabled to help blocking in dark areas and on things like walls in anime. You should probably only select this if you are trying to get rid of a problem you have noticed with an encode you made already.

The last settings we need to configure before starting our First Pass are the Advanced Settings. Click the Advanced Settings box underneath the Zone window and you will see something like this:

The first tab is the Motion tab which allows us to alter how xvid calculated things that are moving in the video.

Motion Search Precision: This tells XviD how hard it should look to see if parts of the video are moving around. You will want to keep this set at 6 for maximum quality.

VHQ mode: This is a new addition to the code in the last year that goes into even greater detail when analysing motion. It's stable now and higher values will increase quality at the cost of speed. You should set this to 4 - Wide Search.

Use Chroma Motion: This is yet another motion searching option that will also take into account the chroma information when deciding how things have moved. This is another recommended setting.

Turbo ;-) This ditches certain mathematical functions to speed up the encode. Not advised, we are in no hurry.

Frame Drop Ratio: You do not want to change this. Unless you really really know why you should, you don't want to change this value. Keep it at 0

Maximum I-Frame Interval: Like previous versions, this sets how far apart keyframes are allowed to be. This should be set to around ten times your video's FPS, hence the 240 here.

Cartoon Mode: Now you are probably thinking that this is a great option for anime encoding. Well actually, it's not really. It's designed for sources made with large blocks of colour and simple lines. The reality is that most anime have all sorts of details in their backgrounds and so on that they don't gain much benefit from this (in fact it can make them look worse). This is for simpsons-style animation not bebop-style animation. Hence, this option is NOT recommended so keep it unchecked. The way it works, for those interested, is twofold. Firstly, it ignores more changes in parts of the picture that are still as they are more likely to be noise and secondly it drops data in low quantization  areas which gets rid of noise but also fine details. The latter is not a good thing to use on most anime and the former can easily be achieved by using a temporal smoother to pre-filter your source.

Once you have set up the options as show above click along to the next tab, Quantization.

This menu allows you to select what range of Quantizers are to be used in your XviD encodes. The default settings have all Minimum quantizers set to 1 and all maximum set to 31 - this is the full range. A Quantizer used by a frame will usually dictate the quality, with 1 being best and 31 being worst.

The old XviD used to have the default minimum set at 2 and this is the option I suggest. Although quantizer 1 is the mathematical highest quality, it is not vastly better quality *visually* than quantizer 2... but it is MUCH bigger in bytes.

As XviD is used mostly for Twopass encoding, quantizer 1 was added to defaults to allow people to fill up all the bits when fitting in a certain size such as a CD. For amvs this isn't really required as every bit saved by using quantizer 2 instead of 1 is precious - especially if you host your own videos.

These settings will also mean that our full quality first pass will be constant quantizer 2 just like the old XviD.

Trellis quantization uses a neat bit of mathematics that simplifies certain parts of the Discrete Cosine Transform and gives bits to others etc etc... in practice this settings pretty much always increases quality, so it's worth selecting. This function was broken but is fixed in 1.0.1

The final tab in these Advanced Settings is the Debug tab which looks like this:

This tab shows you a few things, most of which you don't need.

Firstly there is the CPU optimizations which are automatically detected. The ones here tell smart folk that I am using an Athlon XP. You can choose Force optimizations if you really want but ones that your chipset can't do wil only cause crashes and removing optimizations is only useful if you are getting some odd instability.

Number of threads is for you lucky multi-processor people.

FourCC used: This selects what Four Character Code is put on the avi. This code tells whatever program you use what codec it needs to play it. The default is XVID but as our encode is DIVX5 compatible you can select DX50 so it works on machines that do not have an XVID decoder installed. DIVX works too but people with DivX 4 installed will probably have trouble playing the files.

OutputDebugString debug level: Do you know what this means? If you do I have no more to tell you, if you don't then don't worry about it. Leave it as it is.

Print debug info on each frame: This will dump all sorts of programming info on your video, which you probably don't want.

Display encoding status: XviD has a new encoder window that gives all sorts of clever info about what has been encoded so far. If you don't like it, you can turn it off here.

All done. Press OK to go back to the main page.

Before finalising everything and saving the first pass, let me explain what all the Zone options do and why you'd use them just in case you want to give these a try. On the main page you can see that there is a Zone starting at frame 0 that has a "weight" of 1. What you can do is add a new zone or edit an existing one. Click add to try adding a new zone then click zone options... to bring up this menu:

Start Frame # sets the first frame of the zone. This might be the start of a bumper that you want to have a certain quality or the start of an important hard to compress scene that you want to make constant quantizer 2. There are all sorts of uses for zones as you will see in the descriptions of these options.

Weight:  As mentioned earlier this is a number that alters the quality relative to the overall quality of the encode - so 0.5 would be half the usual bits and 2 would be double.

Quantizer: If you don't want to set the quality by weight you can set it by quantizer. 2 is typically "best" although 1 is better but much larger. Lower values are smaller and worse quality so you may want to use those in scenes or areas which are bitrate hogs but you don't really need them to look good - such as scenes of TV static and so on.

Begin with keyframe: Forces a keyframe at the Zone start frame. This is a useful way to make sure that a keyframe is added in certain places - something we had to edit the stats file to achieve in the past.

Greyscale encoding: tells XviD to ignore all colour information in this scene. This is usually added so people can compress things like end credits even more by removing their colour. You won't need it for greyscale scenes in an amv as XviD can encode these perfectly well without this option - it's just there to remove colour for you.

Chroma optimizer enabled: This is designed to reduce that stepping effect you often get on gradients. This option is not default because it usually increases the size of a video and make it mathematically noisier - however if you see this stepping effect then this option is what you need to eliminate it. It should only be used on a scene by scene basis.

BVOP Sensitivity: This allows you to force an area to have more or less B-Frames. Positive values increase and negative values decrease the chance of B-Frames. It's a sensitive option so 5 will have a decent boosting effect, for example. You wont want to go higher than 20 really and anything lower than -30 won't be noticed and you are not likely to have B-Frames in that section with a value so low. Using a zone to remove B-Frames can be a good thing, especially if you have one or two scenes which contains the B-Frame artefact problem that I will describe at the end of the First Pass.

Remember when setting zones that a zone will either go to the end of the video or until the next zone, so if you want to change something in the middle of your video then you should add two zones with the second being where the video returns to normal. Once you are done setting up zones (if you are setting up any at all)  you are ready to go.

That's all the settings you need for your First Pass. Now it's time to make the encode.

Saving the First Pass

The Second Pass

AbsoluteDestiny - May 2004