What filter do I need? Is this artifact halo?

[mikansei]

>Cannonaire
I cannot get CanderainE to work. It gives me the following error (in AvsPmod):

Script error: there is no function named "tcanny"
(D:\xxxx\CanDerainE.avs, line 38)
#line 38: does it refer to canderaine script? if so this is line 38:
#\ .mt_binarize(threshold=rthr) #from "Create masks" section
(sample.avs, line 27)

I'd like to see how the video looks like with the filter, plus I'll probably use it even if it runs slowly...
You said Derainbow creates a lot of artifacts... What kind? Just to check out because I didn't notice any :S

Then I will proceed to do the tests!

[Cannonaire]

You're missing the required plug-in 'tcanny'.
You can find it on this page: http://web.missouri.edu/~kes25c/

[Mister Hatt]

I have no idea where the guide pulled a lot of that shit about 'preferred usage' of my AA filters from. They were ALL written for dealing with a rather heavily ringed DVD which was aliased as a biproduct of unavoidable cleanup. None are made for old anime or for HD anime upscaling. They are best used as cleanups for Camembert. Naa() specifically was written purely to be a more preservative version of maa() at the cost of speed. Supaa() and splinaa() were written to handle smooth curves better than naa() did and tend to be a little sharper. Sharpaa() was written to be overly sharp at the cost of detail on blurry luma-high scenes. I am assuming I remembered my reasoning for each one correctly of course as that's all off the top of my head, but it should be right. They MAY do other things, but those are the intended effects and usages. If anyone cares about the source in particular, it was the Hyakko OVA with the 60fps ED because I got lazy.

[mirkosp]

You called it shaarp, not sharpaa.
Anyway, even if naa was designed for that kind of aliasing, it does work well on almost any kind of aliasing so I don't see why it shouldn't be suggested.

[Mister Hatt]

I just didn't like the assertion that the particular usage mentioned in the guide is what it was made for when nobody besides myself, TheRyuu, and two others would even know. It might be good for that but it still had an intended purpose. Also you're probably right, I haven't actually used them in like a year so it makes sense that I'd forget the names :3

[mikansei]

>Mister Hatt and mirkosp's conversation
lol
Cool that you mentioned all the AA you created. I just can't find information about them anywhere Where can I download them from?

>Cannonaire:
Thank you, I finally got Canderain working!
I tried the script with CandeRain and in my PC it's also slows down the script a lot
Please, if you have time, could you tell me which artifacts where you referring to when you told me Derainbow makes lots of artifacts? I would like to know to value if I'm going to use it or if it's better to leave it out. As far as I can see I cannot appreciate any artifacts.

This is a comparison I've done applying both filters to this image using the tweak option you gave me for CanDerain (I put this one because I checked out the CanDerain script and tried tweaking it further but since I don't know what each tweaking option is for I didn't get any results ).


I also have doubts about NTSCSquare():
I'd like to know the functions of the filter: It's a filter for converting NTSC DVD to avi/MP4 but also does a bit of cropping. Is that right?

If I want to convert a PAL DVD to avi/MP4 I should use Spline34resize, for example. Or do you have a PAL version?

Also, NTSCSquare the resulting resolution is always 600x480 resolution or can I choose something else?

Well, in my script it gave 600x476. I guess those lost 4px are the 4px cut in the crop filter and would like to know if that affects the image when watching it.
I had always put cropping before resizing but checking your NTSCSource script I've seen that the original video needs to be 720x480. So I did a test and put the cropping before NTSCSquare and it did give an error. So 4px is little cropping... but if I should do a 16px cropping, let's say, then won't the image get too deformed?

I have been doing several tests with the script and this is what I have decided to use:

Code: Select all

MPEG2Source("C:\Documents and Settings\hperis\Escritorio\AvsPmod_v2.0.6\azuki_fragment.d2v")
TFM(order=-1, field=-1, mode=1, PP=0, chroma=false)
TDecimate(mode=1, cycleR=1, cycle=5, rate=23.976, hybrid=0)
DeRainbow()
mergechroma(awarpsharp2(depth=32))
MCTemporalDenoise(settings="low")  
LSFmod(100)   
NTSCSquare()     #resizing filter
Crop(2, 4, -6, -0)      #Minimum you want to crop
maa()

Although dehalo_alpha is really cool and takes away the dehalo_alpha, it also changes the relief of the image, kind of flattens it. I more on the preserving detail and artifacts side rather than on the degraining side, I guess.

>mirkosp:
Thanks for the suggestion about using TTempSmoothF. I didn't know that filter. I want to do a few more tests for the grain. I want to compare your suggested filter and a few others with MCTemporalDenoise.

[Mister Hatt]

TTempSmoothF is actually one of my tricks that I showed mirko, he is my Supreme Itarian Student. Trufax. It's a light weight fast temporal smoothing filter, basically analyses frames in a scene and appropriately cleans while preserving the temporal data or something. I think I might be super high right now so yeah might be wrong idk. dehalo_alpha is imo quite overkill and smooths the shit out of things, better to try like camembert().naa().ttempsmoothf(maxr=2) or something.

[Cannonaire]

I more on the preserving detail and artifacts side rather than on the degraining side, I guess.

That's actually a really good attitude as a lot of people around here tend to over-filter things, myself among them sometimes (although I try not to). I've found that dehalo_alpha can really blur stuff, but in this source I'm actually seeing more detail come through after. Some people think the halos make the image look better because the image does seem to have more depth, but that is certainly not how the image was intended to look.

Derainbowers:
DeRainbow() causes huge temporal chroma issues. I'm not exactly sure what to call them, but let me post a couple screenshots.

Spoiler :

No filters:

Derainbow(10):

Derainbow(10) on your footage (frame 130 of the sample):

Note the huge discoloration around the shield and lower body of the robot. This is temporal chroma bleeding I believe, but my terminology could be wrong. It is most apparent in scenes which are mostly static but also have moving objects, such as this one. You can see the same problem in your footage under the mouth and above the shoulders and a few other places.

CanDerainE() is still in development and experiomental. It has worked pretty well for me on a few older sources, which is why I recommended you give it a try. In order to write what I have so far, I studied 8 or 9 different derainbow filters. The E at the end denotes edge-only, as opposed to an earlier edge and luma version which mirko tried out. I'm open to any suggestions on how to make it work better/faster, and I'm gonna start with ttempsmoothf next time I work on it. Also, I've never used ttempsmoothf before so hopefully it can replace mctemporaldenoise for me. Sorry the filter is so slow, but I rather like the results, especially on the fence in the school screenshot. CanDerainE won't fix everything, but it tends to fix a lot without adding visible artifacts.

NTSCSquare:
Leaving your footage at 720x480 without setting DAR will give you incorrect image geometry (e.g. your circles will be oval), as will cropping 16 pixels and resizing to 640x480. NTSCSquare is a simple script which takes NTSC DVD content and resizes it from 720x480 with rectangular pixels to 648x480 with square pixels. I wrote it to make resizing footage correctly easy (correctly if you don't set DAR when you mux at least), since telling people how to do it didn't go very well. I don't see how it could possibly change the width to anything other than 648 or 864, since those numbers are hard-coded into the script as the only width parameters. "NTSCSquare()" as written should be exactly the same thing as putting this in your script:
Spline36Resize(648, 480, src_left=4.575, src_top=0, src_width=710.85, src_height=480)

I'm not quite sure what would cause it to give you 600x480 unless you changed the script or you're cropping a lot after. Also, DGIndex read your sample as NTSC, and the framerate does not indicate PAL. I hope this didn't come across as angry or annoyed at all, since I'm just trying to help you out.

[Scintilla]

Just a quick note:

Note the huge discoloration around the shield and lower body of the robot. This is temporal chroma bleeding I believe, but my terminology could be wrong.

If this is Shodan's DeRainbow/RainbowKiller/SDDeRainbow we're talking about, the chroma gets spatially blurred a few times before the temporal softening; on the sources on which I've used it, it's always looked like more spatial chroma bleeding than temporal.

[mikansei]

About the chroma halo… I've trying different options and I've decided to use 2 lines instead of all 6. These two lines take away most of the halo and the image gets barely masked, while using all 6 lines makes the image look too masked IMO

Code: Select all

dehalo_alpha(darkstr=0,brightstr=0.6,rx=2,ry=2)
dehalo_alpha(darkstr=0,brightstr=0.8,rx=2,ry=2)

>Mister Hatt:Thank you for suggesting camembert().naa().ttempsmoothf(maxr=2). I tried it out but the halo remains intact with that syntax as far as I can appreciate. Maybe this footage's halo is kinda different from regular halo…? :S


>Cannonaire:Thank you for explaining! I didn't realize the chroma bleeding was caused by Derainbow… I checked out the video
and yes, a lot of the chroma bleeding is caused by Derainbow, although not all of it. Do you think it's better to use CanDerain even though it's slower, or should I use Derainbow? (As far as I could see, using mergechroma(awarpsharp2(depth=32)) afterwards fixes the chroma bleeding.

About resizing, I've always believed the image has to be resized to 640x480. So how is it that the script set up to 648x864?

About the PAL thing, I meant asking if I could use the script for resizing PAL DVDs, but I see it's just for NTSC DVDs, so nevermind.

"NTSCSquare()" as written should be exactly the same thing as putting this in your script:
Spline36Resize(648, 480, src_left=4.575, src_top=0, src_width=710.85, src_height=480)

Thank you for pointing it out, but I don't understand the script: why left and width have those values. T_T I'm confused XD

No, I didn't change the script… I have done a test just using NTSCSource, not cropping, and then I got 640x480 as the output resolution - I got 640x476 when I cropped after using the filters you suggested:

Code: Select all

NTSCSquare()     #resizing filter
Crop(2, 4, -6, -0)      #Minimum you want to crop

So using NTSCSource, since you're cropping after resizing, I should crop as little as possible to preserve the geometry of the image?

What happens the other way round?: If you crop a lot before resizing and then you resize using Spline36Resize(648, 480) the geometry remains?

Code: Select all

Crop(10, 10, -12, -4) #Is this a lot? If not, just put a big enough cropping value.
Spline36Resize(648, 480)

Maybe I should open a new thread for this question, just let me know!

I have this doubt from reading the guide at http://www.animemusicvideos.org/guides/ ... atios.html:

"You know how I said NTSC DVD resolution is 720x480? Well, that's not entirely true. 8 pixels on each the left and right side of the image are supposed to be just empty space, and aren't really even supposed to even be considered! Got that?(…) A DVD is 720x480, but taking 8 off each side, we get 704x480."

So when we resize down to 600x480… Any resizing filters like NTSCSource or Spline36Resize resize the image taking the input video as 720x480 or as 704x480? (Or maybe that doesn't really matter at all?). And how does doing more/less cropping affect to the output video resized to 640x480?

Hope I explained my doubts clearly enough dispite my confusion XD And thank you again for everything. You're really helping me to understand a lot of things.