Overly saturated colors after compressing, etc.

[DriftRoot]

As predicted, my attempts to create an MPEG-2 file for convention purposes crashed and burned. Look what it did to my AMV!!



Everyone has sunburns!! All the colors have become highly saturated, to the point where sometimes it hurts my eyes. It's HIDEOUS!! I've compressed with Xvid and this didn't happen, only MPEG-2. I've tried it in both YV12 and RGB colorspaces, same exact results. I've tried it without any filters, same results. It's the MPEG-2 that seems to be causing this. And if you think the MPEG-2 version looks better than the original, then I give up. There's certain things in my AMV that look terrible in a highly saturated state and I do not want my AMV looking that way PERIOD.

My video is also a little squished. Look how plump Sephiroth appears! I swear I did everything right. It only squishes on me when I run it through TMPGENC Plus 2.5. It's fine, otherwise. I did NOT mess up from the get-go, this time, my uncompressed footage is in a nice 848 x 480, 24 FPS, RGB, progressive state.

The script I'm using for MPEG-2 is as follows:
ConvertToYV12()
temporalcleaner()
msharpen(threshold=50,strength=50,highq=true)
LanczosResize(720,352)
AddBorders(0,64,0,64)
AssumeFPS(23.976,true)
SSRC(48000)

For non MPEG-2 (and it looks great) I'm using:
ConvertToYV12()
temporalcleaner()
msharpen(threshold=50,strength=50,highq=true)
LanczosResize(480, 272)

What the heck is going on? If this always happens to people's MPEG-2 footage, why isn't there some kind of giant, red flashing warning sign somewhere so posts like this don't have to be made?

[BasharOfTheAges]

The color difference does look obvious - not sure about that one. Might be a limit of the encoding software. As for the other part... list for me please... source PAR, project settings PAR, export PAR, and the PARs of each of those. NTSC-speced MPEG-2 should be 0.9. If you edited/exported with 1.0 that would explain the difference.

[Zarxrax]

did you compress this with tmpg? Theres a setting in it that messes with the contrast and stuff.

[Qyot27]

My video is also a little squished. Look how plump Sephiroth appears! I swear I did everything right. It only squishes on me when I run it through TMPGENC Plus 2.5. It's fine, otherwise. I did NOT mess up from the get-go, this time, my uncompressed footage is in a nice 848 x 480, 24 FPS, RGB, progressive state.

The script I'm using for MPEG-2 is as follows:
LanczosResize(720,352)
AddBorders(0,64,0,64)

For non MPEG-2 (and it looks great) I'm using:
...
LanczosResize(480, 272)

Are you viewing the MPEG-2 directly in Windows Media Player, or in Media Player Classic? The reason I say this is that telling the script 720x352 is to adhere to spec and allow it to compensate when the DVD player (or other software which respects AR flags at all times, like MPC) plays it back as 4:3 instead of 3:2. Some decoding filters, when used in the original WMP, will only go 4:3 on fullscreen, and be 3:2 the rest of the time. 480x272 is a square-pixel 16:9 ratio, as would be 640x360 (352 to avoid macroblock inefficiency with non-mod16 resolutions, it's not a *pure* 16:9 ratio in that sense but it's close enough) - but using 640 and pillarboxing the width to 720 would make it look wrong on a DVD player.

[DriftRoot]

list for me please... source PAR, project settings PAR, export PAR, and the PARs of each of those. NTSC-speced MPEG-2 should be 0.9. If you edited/exported with 1.0 that would explain the difference.

Source is .9. I resized up to 1.0, edited at 1.0, exported at 1.0 and then followed Scintilla's instructions for resizing my footage with TMPG (used his OtakuVideo template, btw) and voila, squish. As Quot27 says, it seems to be that 3:2 factor. I had a bad feeling about that to begin with...however, I can't believe I'm the first person this has happened to. This is a HUGE problem.

I've been viewing the footage on a variety of players, WMP, VLC, Nero Showtime, VDM because I knew that could affect my experience. The screenshot of my MPEG-2 is from VLC, the original screenshot is from VDM. It's the same across the board. Even within TMPG the footage is very obviously squished. But even if it is a function of the player, won't this look the same way when the convention entry reviewers sit down to watch it? Talk about instant disqualification. @_@ And the color! UGH!

did you compress this with tmpg? Theres a setting in it that messes with the contrast and stuff.

Yes, I used the Pegasys version. Which setting is it, exactly, that I need to UNset to return my carefully filtered footage to its original state? I see the contrast and stuff adjustments that can be made under the Advanced tab, but none of them are checked off.

Speaking of which, my MPEG-2 overall is really, really ugly. Above and beyond the color problem, it's grainy and blurred (ruined all my careful filtering and vector art) and also seems to have some subtle interlacing going on. This is not the quality I would expect of an MPEG-2. I wouldn't even expect this of Xvid.

I would really like to submit this AMV at the highest quality possible, but something's definitely wrong, here. Is it the Pegasys software? Should I try the other encoders out there? I'd like to find someone who's edited at 1.0 with Advent Children and MPEG-2ed it successfully into proper convention format and compare notes.

[Zarxrax]

Under quantize matrix tab, there is a setting called 'output yuv data as basic YCbCr'.
I think it should be not checked. I haven't used tmpg in a long time though.

[DriftRoot]

Under quantize matrix tab, there is a setting called 'output yuv data as basic YCbCr'. I think it should be not checked. I haven't used tmpg in a long time though.

I remember seeing that, but I do not remember if it was checked. I'll have to look into it when I get home.

list for me please... source PAR, project settings PAR, export PAR, and the PARs of each of those. NTSC-speced MPEG-2 should be 0.9. If you edited/exported with 1.0 that would explain the difference.

Scintilla's helpful template/script is very specifically changing the PAR from 1.0 to 3:2 (he even says to make sure it does) so...is everything supposed to be squished? If it is, why didn't he warn me?! I am highly paranoid about sending out an AMV that looks awful to me but, theoretically, should like fine for whomever gets it. And seriously, I'm not kidding, the more I look at this MPEG-2 that I got, the more horrible I realize it is on several levels. Something is very, very wrong.

I came up with another fact about my AMV that may be helpful in nailing down the problem(s): the footage I’m using was originally compressed with Huffyuv. Premiere crashes about halfway through the process when I try to export the AMV with this, however, so I had to export it with no compression. This is the only thing weird going on that I can think of, but I’m not sure if such uncompressed video would cause TMPG to choke like this. I’m not getting any errors during the MPEG-2 encoding process, TMPG thinks everything’s ok.

Should I, perhaps, try to recompress my exported AMV in VDM with Huffyuv, then take that Huffyuv file and plug it into TMPG? I can think of a few other things to try as well, but since I’m only about 25% sure of what I’m doing, that isn’t really troubleshooting the problem…it’s more like throwing random ideas at it in a panic. Suggestions for other experiments are welcome.

I've spent almost five months on this AMV...that's going to be five months of labor down the drain if I can't get MPEG to work!! <---- this is me really starting to panic

[Qyot27]

Scintilla's helpful template/script is very specifically changing the PAR from 1.0 to 3:2 (he even says to make sure it does) so...is everything supposed to be squished? If it is, why didn't he warn me?! I am highly paranoid about sending out an AMV that looks awful to me but, theoretically, should like fine for whomever gets it. And seriously, I'm not kidding, the more I look at this MPEG-2 that I got, the more horrible I realize it is on several levels. Something is very, very wrong.

Have you tried playing it back fullscreen, though? And how does it look when played in Media Player Classic? Link to the actively-developed Homecinema fork of MPC.

In any case, the reason for 3:2 is due to being aimed at things like standard TVs (which don't use square pixels; TV 3:2 is roughly the same as computer 4:3), not computer monitors. Inside the MPEG-2 file there should be a flag which tells the player to resize to 4:3 on playback - thus, the discrepancy will be gone when used in a player that respects the flag. DVD player software and hardware, and other media players that are worth their salt will do this, but it may be conditional on using fullscreen vs. viewing in a standard window.

If you want to be risky, you could try anamorphic encoding (though I'd suggest asking the convention about this first if not clear about it already; for home DVD authoring I'd recommend this as a priority over doing the 4:3 letterboxing tango). My workflow for anamorphic video is thus:
1) Resize to 720x480 (other ratios may need to be letterboxed to 16:9 beforehand; for instance, a 1920x1080 signal can be safely stored as 848x480, and thus 720x480 by extension, but 1920x816 fits to 848x352 - it would get letterboxed to 480 height to match the 16:9 ratio; I was using HD resolutions as an example since it's easier).

2) Encode as usual with TMPGEnc, making sure that only the video gets processed (I use AC3, which I encode separately), or at least that the streams are not muxed together. This can be achieved with the 'Output audio and video as individual elementary streams' option on the last page of the wizard, or for just the video stream, by additionally going through that, starting the encode, stopping the encode, using the Load button on the bottom right to select the 'unlock.mcf' template, going back to the main screen, choosing 'Video only' under the Stream Type box, and then restarting the encode.

3) Use ReStream to adjust the file's AR flag to 16:9.*

4) Use TMPGEnc's MPEG Tools (Simple Multiplex tab) function to mux the streams back together.

5) Author DVD as usual.



*If you're really paranoid about the current encode you have, you can test it by using TMPGEnc's MPEG Tools (the Simple De-multiplex tab this time), outputting the video stream, and loading it into ReStream. If the Aspect drop-down says '4:3' then you're good and the flag is correctly set - nothing to worry about.

[DriftRoot]

I have played the video back full screen, it makes no difference. Both those screenshots I posted were from full screen. I thought I did have DVD software on my computer, that's what I thought my Nero Showtime player was, since it can resolve DVD aspect ratio issues WMP and sometimes even VLC can't - although TMPG doesn't seem to think I do have the right software (it actually told me I didn't...weird, it got all communicative all of a sudden).

Awesome advice, though, I will try that as soon as I can and report back!

Even if I can resolve the aspect ratio discrepancy, the video quality/color shift is still ruining my AMV. I did a little digging, though, and I think I found someone who experienced the same/similar color shift problem I did with MPEG-2. I don’t understand half of what’s in this thread, but the problem sounds disturbingly familiar and the images he posted look even more familiar – sunburn central.

They seem to recommend using a filter called ColorMatrix to eliminate the color shift. I guess I should give it a try, unless you folks think it's not worth it. If that thread gives anyone with a lot more knowledge than me any ideas about how to correct my color problem, I will be extremely grateful.

[Zarxrax]

Colormatrix can help with color problems, but thats for the source itself. If the problem is occurring during compression, then its a problem in the compression, not the source.