What filter do I need? Is this artifact halo?

[Mister Hatt]

If AA is needed, might be worth trying one of my Hyakko OVA anti-aliasers? As for the ghosting or whatever, I am unable to download your link, see what mirko or Cannonaire think of your footage. I would also try Camembert() on your haloing although it might flatten other parts of your image. Granted it won't smooth the fuck out of it like that dehalo_alpha() is bound to do, but it will certainly make aliasing more noticable. Over all I think you would have more detail via Camembert() though either way.

[mirkosp]

The antialiasing there was just an example (and it's actually pretty close to what your naa does, just with nnedi3 instead of nnedi2 and without the _rpow2 upsampling). I wanted to make clear what avisynth uses as last and it was the first thing that came to my mind.
As for checking the source, I'm kind of tied up atm. I could dl and watch it, but it'd be a on a 6bpc monitor and I don't actually have avisynth at hand, so I wouldn't be able to help much right now...

[Mister Hatt]

I have 8bpc here but my internet blows and my res is tiny, so not much help either. Way to go us, cue Ryuuchin or Cannonaire? ww

[Cannonaire]

I'll take a look at it. I can't seem to find a link for Camembert(). Anyone able to provide one?

[Cannonaire]

This is how I would filter it:

Code: Select all

TFM(order=-1, field=-1, mode=1, PP=0, chroma=false)
TDecimate(mode=1, cycleR=1, cycle=5, rate=23.976, hybrid=0)
CanDerainE(str=255 , dstr=80, thr=2.0, radius=6, rthr=90, mask=false)
dehalo_alpha(darkstr=0,brightstr=0.6,rx=2,ry=2)
dehalo_alpha(darkstr=0,brightstr=0.2,rx=2,ry=2)
dehalo_alpha(darkstr=0,brightstr=0.8,rx=2,ry=2)
dehalo_alpha(darkstr=0,brightstr=0.8,rx=1.2,ry=0.45)
dehalo_alpha(darkstr=0,brightstr=0.4,rx=1.2,ry=0.45)
dehalo_alpha(darkstr=0,brightstr=0.6,rx=1.2,ry=0.45)
dehalo_alpha(darkstr=0,brightstr=0.4,rx=0.45,ry=2.0)
mergechroma(awarpsharp2(depth=12))
MCTemporalDenoise(settings="medium")
LSFmod(100)
NTSCSquare()
Crop(2, 4, -6, -0)
naa()

NTSCSquare: http://pastebin.com/AtDsyN1U (shamelessly linking my own script for simplicity's sake)
CanDerainE1: http://pastebin.com/X1GwkqWh (again)

You will probably want to adjust the crop() to something mod16, or at least mod8 (meaning the resolution can be divided by 16 or 8, respectively); the values given are a minimum by which you should crop. As for MCTemporalDenoise, it's a total pain to get it working and it also runs really slow, which is bad cause the script is already really slow. If you need to remove the noise, there are many other filters you can try instead. This page is a good resource for figuring out how to denoise: http://www.aquilinestudios.org/avsfilters/spatial.html

Screenshots:

Spoiler :

Group unfiltered

Group filtered

Spoiler :

Girl unfiltered

Girl filtered

Spoiler :

School unfiltered

School filtered

Also, thanks to mirkosp for the tip on using multiple dehalo_alpha() instead of just one. You have helped me tremendously!

[Cannonaire]

Okay, sorry for the triple post, but I just tried rendering the thing. It looks really nice but it renders at about .15~.21fps , which means it takes about an hour for less than 600 frames to render. It isn't MCTemporalDenoise causing that, so my guess is that using all those dehalo_alpha lines, while ideal, is causing a massive slowdown. I'm kinda tired right now, but I'll work on it again later.

Also, someone gave me a link for camembert(), which didn't work well in this case.

[mikansei]

Thank you all for your help!! Yesterday I didn't have time to check out your answers, so today I'll check everything out and get back to you.

Here's the link to camembert: http://mf.creations.nl/avs/functions/Camembert-v0.1.avs

[mikansei]

If AA is needed, might be worth trying one of my Hyakko OVA anti-aliasers? As for the ghosting or whatever, I am unable to download your link, see what mirko or Cannonaire think of your footage. I would also try Camembert() on your haloing although it might flatten other parts of your image. Granted it won't smooth the fuck out of it like that dehalo_alpha() is bound to do, but it will certainly make aliasing more noticable. Over all I think you would have more detail via Camembert() though either way.

>mirkosp
Thank you very much. Gotta study some syntax!!

>Mister Hatt
I tried the link for the video and it works. If you can check it out from another PC, please post later

Didn't know the naa() filter was yours. Cool!
The AMV guide says that the naa() filter is to be used for HD sources that have been upscaled and that I should use daa() for aliasing from combing. My DVD is an old anime (not HD and is hand-painted), the original resolution is 720x480 and I want to resize it to 600x480 (I read you have to resize to this resolution NTSC DVDs, right?). So before posting here I was trying with daa(). So can I also use naa() in old anime?

>Cannonaire
Don't I need to add the spline36resize for resizing the video to 600x480? :O
1hour for 600 frames is long indeed

As Cannonaire, I also tried using Camembert() but I got no visible results. I mean, I couldn't appreciate any change in the image

I haven't had time to test the scripts yet. I'll try to find time to do a video test within today and thursday using dehalo filters and comparing it to using a script without the dehalo filters. I also want to try the script Cannonaire gave me (thank u!) and do some comparations with my script. Will post back!

[Cannonaire]

>mikansei
-NTSCSquare is a simple script I wrote to correctly resize NTSC DVDs. After using it, there is no need for any more resizing, but cropping is often still necessary after.
-I also did not get good results with Camembert(). It is not needed for this footage.
-I rendered using exactly the script I gave you after my source line (except I had a modded version of naa() which doesn't really matter). It looks great, but .15fps is far too slow for any practical use. (note that any frames-per-second numbers are only for my system, yours may be faster or slower)

I just ran a bunch of tests removing one function at a time, and the script runs at about 1fps if you remove just naa(). It runs at about 1.5fps is you remove both naa() and MCTemporalDenoise. Same thing if you remove naa() and CanDerainE(). Removing all three of those filters but keeping everything else makes it render at about 4.5fps...

Because the most obvious problem in your footage is halos, this is good news. You can run all of those dehalo_alpha() lines and fix your biggest problem. As for the other filters, it's up to you. Derainbow() runs just as slowly as CanDerainE(), but it also adds a lot of bad artifacts to your chroma. MCTemporalDenoise() is just a filter I like to use, but it is really slow and another noise reduction filter would probably do alright. naa() looks pretty nice on this footage, but it causes way too much of a slowdown to be practical.

With all of that in mind, here is a revised script. It will work as your entire script if you choose. To use the disabled filters, remove the # at the beginning of the line.

Code: Select all

#Replace this line with your source
TFM(order=-1, field=-1, mode=1, PP=0, chroma=false)
TDecimate(mode=1, cycleR=1, cycle=5, rate=23.976, hybrid=0)
#CanDerainE(str=255 , dstr=80, thr=2.0, radius=6, rthr=90, mask=false) #You decide whether to use this or not.
dehalo_alpha(darkstr=0,brightstr=0.6,rx=2,ry=2)
dehalo_alpha(darkstr=0,brightstr=0.2,rx=2,ry=2)
dehalo_alpha(darkstr=0,brightstr=0.8,rx=2,ry=2)
dehalo_alpha(darkstr=0,brightstr=0.8,rx=1.2,ry=0.45)
dehalo_alpha(darkstr=0,brightstr=0.4,rx=1.2,ry=0.45)
dehalo_alpha(darkstr=0,brightstr=0.6,rx=1.2,ry=0.45)
dehalo_alpha(darkstr=0,brightstr=0.4,rx=0.45,ry=2.0)
mergechroma(awarpsharp2(depth=12))
#MCTemporalDenoise(settings="medium") #Use this or another noise filter that runs faster.  Or no noise reduction.
LSFmod(100)
NTSCSquare()
Crop(2, 4, -6, -0) #Minimum you want to crop

[mirkosp]

I think it might be worth to give TTempSmoothF a try here (do mind the F, it's a faster version of TTempSmooth that skips certain steps which should not really matter in this case)...