Cleaning up script help.. with examples

[Taruto!]

I've been working on cleaning up the Master of Mosquiton OVA for AMV use in AviSynth. Normally I'm alright with AviSynthing footage, I've managed to clean up a few series quite nicely but this series is just giving me no end of grief to clean x.x

Here are some screencaps of the original footage & with my current script.




and this is my current script:

Code: Select all

LoadPlugin("E:\mpegdecoder.dll")
mpegsource("E:\ ... ")
ChangeFPS(29.97)
fielddeinterlace()
tweak(hue=1.0,sat=1.2,cont=1.1,bright=-2)
awarpsharp(12)
Fastlinedarken()
Deen("w2d",4,8,6)
Crop(32,14,-32,-14) 
LanczosResize(720,540)

..yes, the resolution is strange.. but I like it sort of like a step up from 640x480.

In any case; the lines are jaggy and thin, some scenes have some pretty bad dot crawl, some areas are VERY blotchy (see first scripted image: bottom of left side of hair) theres a horrible grainyness all over and it all looks.. washed out? Playing with the colours any more just made the colours look distorted or too bright.

I can't figure out HOW to clean this footage up. My current script is the best I've managed to get it looking. Currently I don't have any additional scripts since they almost always crash VirtualDubMod, so I'm working with the most basic of scripts here, but if anyone can recommend some to try I'm happy to see if they'll work.

Any help is appreciated, this is driving me insane.

[EvaFan]

Well... This could quite possibly be the worst grainy source I've ever seen. You definitely need:
Removegrain(mode=5) (mode=5 is good, mode=17 if you want to be aggressive)

<b>Remember... Removegrain goes before your deinterlacing.</b>

TBH I hate the way awarpsharp looks on anime.
Consider using spatial and temporal cleaners instead of warping, here's my personal favorite:
http://avisynth.org.ru/fft3dfilter/fft3dfilter.html

Make sure you get the file for your system32 folder all so on the page if you plan on using it. It's not hard to learn how to use the filter if you read through the page.

It looks like there is all so dotcrawl on the plane letters. Consider using this which scintilla was so kind enough to tell us all about:
http://www.animemusicvideos.org/phpBB/v ... hp?t=91129

The order of your script is important too. Tweak should be one of the last things you do. put it before your crop. Deen should be after your deinterlacing. I'll post more help later if I can think of anything but the regulars will prolly be posting soon.

[Kariudo]

Well... This could quite possibly be the worst grainy source I've ever seen. You definitely need:
Removegrain(mode=5) (mode=5 is good, mode=17 if you want to be aggressive)

<b>Remember... Removegrain goes before your deinterlacing.</b>

I wasn't aware that removegrain needed interlaced footage...in fact, I've never used it before I IVTC.
mode 2 is a little more aggressive than 5...but not by much. I'd try 2 before 17, and then try

Code: Select all

removegrain(mode=5)
removegrain(mode=5)

*My recommendation (and possibly Eva-Fan's too) assumes you have removegrain 1.0, not 0.9

I hate warpsharp/awarpsharp as well, I'd add fluxsmoothST() to the spatial-temporal smoothers list

I'd recommend indexing your .vob with dgindex and using mpeg2source instead of mpegsource and then changing the FPS (changing the fps of interlaced footage makes me very uneasy. I've never done it, so it may just be a feeling)

There is dotcrawl, though it looks to be manageable. If LUTDeCrawl doesn't work for ya I'd try Tcomb

720x540 isn't mod16, so if you continue with that you may run into problems when you make your final encode.

I hear that warpsharp can break some avisynth things (which, from personal experience, can cause vdubmod to crash) so I'd take it out of the plugins folder

[EvaFan]

<b>Remember... Removegrain goes before your deinterlacing.</b>

I wasn't aware that removegrain needed interlaced footage...in fact, I've never used it before I IVTC.

Gah I read the documentation wrong. I been doing it wrong for a couple months now... Well its good to know now.

Modes 1-10,17,18 are for truely progressive input only, while modes 13-16 are primarilarily for interlaced input.

my bad , thats what I get for speed reading.

[Scintilla]

Thirding removing the warpsharpening, as it's killing shapes (especially with the subpar deinterlacing -- if I remember correctly, FieldDeinterlace is not that good). Yes, the edges need enhancement, and FastLineDarken() is a good first step, but warpsharpening is not the way to go for this source, methinks.

Instead, I'd suggest either LimitedSharpen, MSharpen, or PSharpen (all of which have been written up <a href="http://www.aquilinestudios.org/avsfilte ... l">here</a>).

Also, it looks from the second screenshot like you also have rainbows. (This is very often the case when you have dot crawl.) There are a number of good filters for tackling this problem, and Zarxrax goes over some of them <a href="http://www.animemusicvideos.org/guides/ ... s">here</a>; just keep in mind that many derainbowers (BiFrost, TComb, LUTDeRainbow, etc.) are designed to go BEFORE any IVTC operations.

[Taruto!]

Thanks everyone for your help. I've downloaded all of the mentioned scripts and I'll be experimenting with the footage tonight. I'll post screencaps of the results when I get it looking better

[Taruto!]

Sorry for the double post (once that spam's removed, anyway). Here are the "cleaned" screencaps and my new script. I think it looks a lot better compared to the original footage and it's probably the best I'm going to get considering the original animation quality without having to redraw scenes myself.

There are still a few things I'd like to tweak, but for now I'd say it's a big improvement:





This is my current script:

Code: Select all

LoadPlugin("E:\mpegdecoder.dll")
mpegsource("E:\ ... ")
ChangeFPS(29.97)
removegrain(mode=17)
fielddeinterlace()
Deen("w2d",3,8,6)
limitedsharpen()
lutderainbow()
tcomb()
Fastlinedarken(strength=100,thinning=0)
tweak(hue=1.0,sat=1.2,cont=1.1,bright=-2)
Crop(32,14,-32,-14) 
LanczosResize(640,480)

Is there anything else I can do to improve the quality?

[Scintilla]

LUTDeRainbow and TComb should probably be going *before* Deen and RemoveGrain; smoothing will likely screw up the rainbow detection.

[Kariudo]

It looks like fastlinedarken is set too high on that second screenshot.

using lutderainbow and tcomb is most likely washing away finer details, you should probably stick to tcomb (which can take care of both rainbowing and dot crawl) or lutdecrawl/lutderainbow

IIRC, tcomb needs interlaced footage to work properly...so it should be going before your feilddeinterlace() call
*checked the documentation, and tcomb does need interlaced footage*

[Taruto!]

Thanks again, everyone. I've fixed up the script as you've said and these are the new results, not much different from before but a little improved:





and this is the current code:

Code: Select all

LoadPlugin("E:\mpegdecoder.dll")
mpegsource("E:\ ... ")
ChangeFPS(29.97)
tcomb()
removegrain(mode=17)
fielddeinterlace()
Deen("w2d",3,8,6)
limitedsharpen()
Fastlinedarken(strength=50,thinning=0)
tweak(hue=1.0,sat=1.2,cont=1.1,bright=-2)
Crop(32,14,-32,-14) 
LanczosResize(640,480)

Regarding LUTDeCrawl: I tried using it and it gave me some weird errors, so I used TComb instead.

Though I noticed during playback, the clip has a slight jitter. It's not too noticeable but it's definately there. Is there a script to stop or reduce this in any way? There's an example here showing jitter on scenes (don't mind the compression, it's just an example): mosquitonexample.avi (3,582kb)

If there isn't a way to fix it, that's alright, but I think I remember reading you can fix jitter somewhere? Not 100% but if there is I'm happy to try it.

Again, thanks everyone for the help! It's really appreciated.