ZarxGui making it all worse

[Animated]

Truth is, last time I used Avisynth and ZarxGui was 2 years ago. So, now, when I try to export my Huffy into a mp4 using ZarxGui, in the end I see serious amounts of blocking, banding and grain, especially in darker scenes. No avisynth used, even with the quantizer on 15. I also tried some deblocking scripts and tried to reduce the grain, and even played with different presets and tunes but nothing - and the filesize is always above 100mb. So.. any good advice on this? Thanks in advance.

[Cannonaire]

Dithering and Grain should actually help prevent blocking and banding to a degree, but it does not always work like that. You may want to read about the deblocking settings for x264 here and here.

You might also want to try using the grain preset instead of animation if you have not tried that yet. I had trouble with some dark Cowboy Bebop footage looking awful until I adjusted the settings to --deblock 2:-2, --crf 17 (quantize), and --tune grain. Your results may vary, and you will likely need to play around with the settings until you get decent results.

[Animated]

Yes, I have already used the grain preset but it seemed to me that it... added more grain, not the opposite. And that's for the dark scenes, in shiny coloured scenes there's no problem at all. Anyway, I'll keep experimenting... what else Thank you Cannonaire.

[Animated]

Also... Jesus. I remember that the mp4 I was getting back then was always around 50-80 mb. Now it's always over 150 with the same settings as then, and around 110 if I make the quantizer 20. No script used for a 2 minute video. Something must be wrong (every video I'm seeing on the .org is under 100 mb and superb quality)... is it?

[Mol]

Are You using newest version of zarx?
Also you can try instead of quant using bitrate (lets say around 2000-3000) ,and see what happens.

[Kinematics]

I've done some checking on your video. Definitely doesn't look easy to reduce the file size. Finally found a nice bitrate histogram tool (Bitrate Viewer), so it becomes much easier to see where the issues lie.

Histogram for your video:


As you can see, there are 4 high-bitrate areas. A few seconds each around 20 and 36 seconds (both around 25 Mbps each, avg), and then two longer periods (around 12 Mbps each). 20 and 36 are high-noise scenes, and the two longer periods are the segments with the banner scrolling and staticy video footage.

This type of high-bitrate profile only shows up when there's lots of noise in the scene. Running through a bunch of other contest vids, it took a while to find one that wasn't fairly flat over the entire video. The first one I found to have spikes was "[HaydenST] Time for Life" from Quickening round C-2; it's the TGWLTT video, with lots of 'old film' noisy footage filters, and it's exactly those filtered portions that jack up the bit rate. It was also a very large file, at 116 MB.

While there are certain tweaks you can make in Zarx's package to improve the video a bit, there's not a lot you can do if you want to use those types of noisy effects; they are inherently bitrate-hungry. If using CRF, you can mess with VBV settings to maybe put a cap on bitrate (this is probably the better solution, but it's complicated, and I'm not familiar enough with it to offer any suggestions on how to go about it). Otherwise, if you really want to drop the file size, your best bet is probably a 2-pass encode. Average for your 102 MB encode is 6000 kbps, so you can scale it down proportionately to how large you want the final file to be.

[Mister Hatt]

That tool would be more useful if it colour-coded for frame coding types, and showed what the quantization factor was for them.

There are a few things you need to be aware of in how x264 and DCT compression handles things like blocking, banding, and grain. To start with, you mention setting the quantizer to 15, is that actually using CQ mode or did you mean CRF? CQ will always give you inflated bitrate and shittier results on account of quantizing every frame the same way essentially, where you want to adapt it to your video by either limiting bitrate or otherwise setting a required rate-factor.

The above post is correct in mentioning VBV as an area to investigate but I don't think it will be of much use until you sort out the way you want to set your bitrate constraints in the first place. Now, onto DCT. x264 as you know uses MPEG4-AVC (H.264) which is a DCT codec. What that means is it splits your footage into a bunch of 16x16 macroblocks and then approximates the data for each macroblock. The kick is that it takes a certain amount of data to do that. An approximated value will usually be smaller in required size than the full value which is why it gets compressed, but it will be limited in either the size you allow it to take (ie how good the approximation is) or limited in how good it is directly (ie the quantization factor you specify.)

This is essentially gonna be a writeup on how x264 does stuff and hopefully let you figure it out on your own, but I'm sure others can guide you from here on. When you set a limited number of bits to a frame, x264 has to move bits from one part of the frame to another. In restricted bitrate mode, you can influence this directly. This will result in some parts having detail while others being blocky or flat, and is where the aq-strength (and in a limited capacity, qcomp) come in handy. If you're going with a constant quantizer however, you're telling it to compress everything x amount, and that amount might be fine for some frames and terrible for others, so you end up with blocky shit as well. You can set a really lazy quantizer if you want but filesize will be stupidly high.

As for banding and grain, check if it really is grain, and if there is banding in the source. Banding in the output only is caused by two things. The first is dither in your footage which is hard to compress very well, especially when limited in bitrate. The second is when the frame itself has limited bitrate distribution and you can tune this with AQ again. If using a constant quantizer, you'll have the same situation as the paragraph above. The thing is, x264 reads flat surfaces as no detail. Blocking is always a sign of not enough bitrate but banding can be caused by the source, lack of local bitrate, or an overactive quantizer due to relatively flat surfaces. The way to get around banding is with dither and grain. If you dither too much and lack bitrate, you make it worse. If you add too much grain, it looks grainy. Is your grain actually grain or is it noise? Grain is GOOD for your source. If you explicitly don't want grain (remember that grain tricks the eye into thinking it's extra detail), and you still have banding even with adequate bitrate, run a quick smooth filter over your video to remove the dither. It's really the process of elimination here. I personally think grain is good in animation because you won't get anything out of it otherwise, while noise is annoying. Dither is fine but even if it compresses properly, someone on a 6-bit display (hi Apple users) will most likely see it as nasty banding.

In the end, you need to give it enough bitrate first, ignore the filesize, and see where the problems lie with the banding and grain you speak of. Once you know that, you can cut down the filesize and proceed from there.

tl;dr use CRF 18 with good AQ settings and it should be fine

[Animated]

Wow, that was really detailed. Too much information I didn't even think about... Thanks a ton guys, how could I ever express my gratitude? (most helpful members on the next VCAs maybe...)

So, I'll just keep experimenting having in mind what you all told me. Seriously, thank you all guys.
the grainy scenes... at 30.000 kbps... holy...

[Zarxrax]

If you are adding noise and grain yourself, there are a few things you can do to help alleviate the problems.

- First off, if you make the noise less strong, it will compress better. Probably not something you want to do though.
- Also, trying different types of noise effects may work better than others. Depending on factors like the size of the noise, it can effect how it compresses.
- Next, there is an avisynth filter called "GrainOptimizer" that can retain grain while making it use less bits: http://forum.doom9.org/showthread.php?t=130611
- Finally, I have discovered that by using a repeating noise pattern, you can reduce the bits needed to encode it. The more frames you have in the sequence the more natural it will look, but fewer frames in the sequence compresses better. By using a 6 frame repeating sequence on a test clip, I saw a bitrate reduction of about 40%, though the repetition was noticable. By introducing randomness into the sequence it can vastly improve the appearance, and also changing up the pattern at scene changes or in high motion scenes can go a long ways towards keeping up the illusion of random noise. Adding grainoptimizer on top of this would probably make it look even more natural while reducing bitrate even further.

[Mister Hatt]

Using static grain is a good idea however it tends to look awkward if you have too much of it. In an AMV you could get away with static-per-scene, and only use it on some scenes and very lightly as Zarx has said. The constant=true param on various gradfun filters will do this. Another one to look at is my favourite grain filter, grainfactoryv3. It adaptively adds grain only to small gradients as dither, and otherwise only to dark scenes as fake detail, matching it perfectly to what x264 does well.

Dark_Shikari's filter linked above is very good but IMO unsuited to animation. It is designed for improved inter-coding of video with large amounts of grain already. This mostly makes it useful for films, and especially so when coupled with the fgo patches to x264. Animated content tends to be either noise (which you should get rid of) or grain that you add yourself in a reasonable manner, so I don't think it's a very relevant filter for this use-case. That said, if you're encoding Mouryou no Hakko or Phantom then you may want to look at it anyway.