Artifacts in Anime/Toon Footage

[Mister Hatt]

The sigmas in frequency domain filters are the frequency sums, yes. DFTTest is a somewhat global sigma, while FFT3D lets you define smoothing strengths for various sigmas. The default in FFT3D is for larger blocking, about what you'd be throwing Deblock_QED_MT2 at. Sigma2 is for slightly smaller blocks, common in poor quantization. Sigma3 is probably the one you want; it's for stuff that is dirt or larger grain sized. Sigma4 is for mosquito noise. I guess something like FFT3DFilter(sigma=0.6, sigma2=1.2, sigma3=2, sigma4=0.4, bt=4, plane=4) should so it.

Contrasharpening is a method of smoothing detail as little as possible on edges, then masking it and applying a difference mask to the unfiltered video. It basically cleans up blocky edges. You can get it here. EdgeCleaner is probably giving you the error because you don't have awarpsharp loaded. You might also have a version it doesn't like, and tbh if that happened then you get what you deserve for being dumb and auto-loading standard filters which should NEVER be done.

I don't think you need Deblock_QED at all really. I'd just find a lowish sigma and dft nuke it or something. If you really care though, your best bet is probably a two-stage FFT3D followed by some EE or something just to clean it up a bit. I would avoid lsfmod but my reluctance to use that filter is based more on a personal dislike for it than any justifiable problems it has.

I just checked my script, it is 4091 lines long but I hope you don't think I wrote that manually. It was created with YATTA of course; how else would one make black magic so agreeable?

[Anno-san]

Here is a better sample of the footage.

[Mister Hatt]

Direct link please. Or someone else like mirko can take a look. This isn't really interesting enough for me to go out of my way and follow crappy-download-provider-x links.

[mirkosp]

I'm downloading it, but it has no extension. Is this an m2v or what? I'll reupload it to voodoo-powered with an extension, so Hatt won't have excuses to not take a look at it.

[Anno-san]

I'm downloading it, but it has no extension. Is this an m2v or what?

Source VOB.

[mirkosp]

http://voodoo-powered.net/~mirkosp/ed%2 ... %201_1.VOB ok, there we go.

[Anno-san]

Yes. That sample has the scenes that I took the images from. It will be much easier for others to test out the filters that way.

[Anno-san]

Has anyone tried a script on the new sample to eliminate the chroma jaggies?

[mirkosp]

I actually didn't get any chroma subsampling error with the source... even if I just try to playback the vob it looks fine (well, as far as subsampling goes, duh ). I didn't post any earlier because I was hoping for Mister Hatt to have a better insight on what might causing the issue on your end, but he's having some issues with some of his software atm from what I understood. As for me I don't really know what else to say. There probably is something wrong with your playback setup but I can't quite put my finger on what since it's not about the renderer itself, apparently.

[Anno-san]

I actually didn't get any chroma subsampling error with the source... even if I just try to playback the vob it looks fine (well, as far as subsampling goes, duh ). I didn't post any earlier because I was hoping for Mister Hatt to have a better insight on what might causing the issue on your end, but he's having some issues with some of his software atm from what I understood. As for me I don't really know what else to say. There probably is something wrong with your playback setup but I can't quite put my finger on what since it's not about the renderer itself, apparently.

Does it matter if the processor is 64 bit but I am using the 32 bit version of AviSynth and all the filters? Is it some problem with the graphics card? It's an ATI Radeon 3200 HD.