Artifacts in Anime/Toon Footage

[Anno-san]

@mister hatt

Code: Select all

Dgdecode_mpeg2source("C:\Users\?\dvd rip\Ed, Edd n Eddy\Ed Edd n Eddy Episode 1.d2v", info=3)
TemporalDegrain()
dfttest(ftype=1, tmode=0)

This doesn't seem to work. Can you post the script you used for the cobra video?


@mirkosp
using ConVertToRGB32() seemed to remove the jagged parts on red edges. The only problem is, when I open the script and try to queue the job in MeGUI it says that ConvertToYV12() should be used at the end of the script. Is ConvertToYV12() really necessary to encode the video in meGUI? What would happen if I encoded it with ConvertToRGB32() at the end of the script?

[mirkosp]

Like I said... the converttorgb32 was just a proof that the issue wasn't with the video itself, but rather with how the yuv -> rgb conversion was handled. You should NOT have the converttorgb32 line in the script, but should leave the video as yv12 (the input already is) and should just have a better renderer in MPC-HC when playing back your video (if you installed the CCCP, possibly the latest beta, but any version would go, you should have haali renderer already).

[Mister Hatt]

I actually can't post the script due to it being customised for each individual episode, and the fact that it's about 4000 lines long. I can however suggest on how to make a mask. You can use IsCombed() by Tritical (or any other comb/jaggy/edge detection filter really) to create a separate clip, then process that with dfttest on appropriate settings, then merge with the original clip. That's assuming you're dealing with the subsampling issue at least. As for the blocking, I would advise a tweaked FFT3D filter for that. FFT3D uses various sigma's to define a particular frequency. You need to find what frequency your blocks are at and then smooth them with the appropriate strength. Read the manual for info (I'm too tired to explain further - mirko, go say stuff なう) and you should figure it out well enough. Finally, as mirko said, the odd colour changes were due to a poor colourimetry conversion which should never have been done in the first place. Get on IRC in about 8 hours maybe if you want to get something resembling example code out of me.

[Anno-san]

@mirkosp

I have already said, I heard that madVR handles the chroma upsampling very well so I used that renderer.

MPC-HC using Haali renderer

Code: Select all

Dgdecode_mpeg2source("C:\Users\?\dvd rip\Ed, Edd n Eddy\Ed Edd n Eddy Episode 1.d2v", info=3)
TemporalDegrain()

I don't see that that makes much difference.

I don't mean to be rude but it seems that, like in doom9, everyone keeps posting contradictory advice. I have been trying to fix this problem (which may or may not be chroma upsampling error) for days, even weeks if you count doom9. What IS the problem? I just want to restore the video and encode it but unless I want to encode poor quality footage, I can't until someone gets to the root of the problem. The whole point of using DVD footage is to restore the video to as high quality as possible. I tried looking through A&E's guide but so far nothing worked.

[mirkosp]

Well yes, I heard good things about madVR myself, but if you happen to have the bad upsampling during playback with it, then clearly you should try something else. By the way, it doesn't matter if you end up having the upsampling in the screenshots, but just if you have it or not during playback, that's what matters. If you *do* have the bad upsampling during playback too, both with madVR and Haali, then I'm not quite sure what is at fault here...

[Anno-san]

Haali renderer might be slightly better. I can't really tell.

Some questions.

1. If haali eliminated the chroma error should there be no jagged edges at all? I still see some.

2. What would happen if I encoded the video anyway?

3. Am I only seeing the artifacts on MY screen, so that if others viewed the same video on their computers they WOULDN'T see the problem? Or is it an issue with the source file itself?

4.Renderers only affect how the video looks in MPC, correct? Or does a renderer affect how video will look when it is copied to the computer via DVDDecrypter?

[Mister Hatt]

Why do you keep saying you don't know what the problem is and that people are giving you contradictory advice? I've fully explained exactly what it is AND told you what to do about it. Every decent encoder I threw your screenshots to said exactly what I already have. The chroma subsampling error (there is no UPSAMPLING error at all and anyone who says it obviously doesn't know what upsample means) is part of the footage and your splitter choice will not fix it.

Renderers affect how your GFX card outputs each pixel. It has nothing to do with software for the most part. Rather it takes colours, applies display operations to figure out what each should be represented as in output, and then outputs it. It's done mostly on the hardware side by your video driver; however different renderers in your player will send slightly different information to your hardware.

[Anno-san]

@Mister Hatt

So the problem can be fixed with either Deblock_QED etc. or Edgecleaner or another masking filter? I'd prefer to solve the issue in as little lines as possible. If your script really was 4000 lines, I clearly can't do that as I have only been using AviSynth for a > a year.

[Anno-san]

Also, I have Edgecleaner v. 1.03 and when I try to run it it give the error "there is no function called aWarpsharp". Usually this error almost always means that the wrong version of said filter (in this case aWarpsharp) is being used. So what aWarpsharp do I need.

[Anno-san]

The other problem with the red jaggies/blocking on line edges is due to terrible chroma subsampling. The chroma issue can be fixed with EdgeCleaner I guess, although contrasharpening might be a better option.

Edgecleaner did not help much. what is contrasharpening and what filters can do this?

The blocking you need to find the golden settings of whatever your favourite deblocker is but I would consider dfttest masking or highly tweaked FFT3D as another option; I ended up using a dfttest mask on Cobra which had similar issues and it turned out pretty sugoi.

What were you saying about finding the chroma's frequency? Is that the sigma? Basically, will FFT3DFilter fix the jagged chroma edges on red objects or will EdgeCleaner? If FFT3D then what parameters?