vmToon and LimitedSharpenFaster

oside · Post by **oside** » Sun Aug 03, 2008 3:11 pm

Can someone help me with the VMTOON problem, I want to use it but I'm on hold until someone can help me out, I apologize for the double post.

EvaFan · Post by **EvaFan** » Sun Aug 03, 2008 4:45 pm

I remember having this problem before and having to edit the script manually. There was an error on one of the lines. The error was caused by "lutxy" not being labeled correctly or something like that. Sorry I dont remember exactly but i'm sure if you google, you might find it like i did.

oside · Post by **oside** » Sun Aug 03, 2008 5:09 pm

Thank you for your answer. The script that I'm using for the filter is located on the top of this post. When I use it, it doesn't give me any error messages but it turns the image green. Hope someone else can help me out. Thank you!!!

BasharOfTheAges · Post by **BasharOfTheAges** » Sun Aug 03, 2008 7:00 pm

Eva-Fan didn't say it gave an error message, just that something errored. There was probably some call that was undefined or handled incorrectly in the definition of some script call. Have you tried googling it like Eva-Fan suggested?

oside · Post by **oside** » Sun Aug 03, 2008 9:28 pm

Ah ok! I've tried googling but no answer yet, I'm going to keep trying. Can someone providing with a vmtoon script? or Can someone tell me what do download besides the script on top of this post? Hope I can find the answer soon in google. Thank you guys!!

EvaFan · Post by **EvaFan** » Sun Aug 03, 2008 11:42 pm

Well since i cant remember what i did to fix the script you can just have mine if you want it. I use this vmtoon in conjunction with the masktools allso in the zip file.

Delete any other versions of vmtoon you got in your folder. You DONT have to delete any other masktools in your folder unless there is one named the same obviously.

http://www.valkyriestudios.org/evafan/vmtoon.zip

Turns out, i think it wasn't a problem of "lutxy" being labeled wrong. I think it was the coding after it being wrong. I cant find it anywhere and I dont remember how I found it so I'm at a loss. Just download the above and put em in your plugins folder.

oside · Post by **oside** » Mon Aug 04, 2008 3:06 am

Thank you Eva_Fan. I already replaced the masktools and warpsharp dlls on the avisynth plugins folder, but I have an error popping out whenever I try to use vmtoon on VDM, this is the error that comes out:

Avisynth Open Failure:
Script Error: there is no function named "mt_lutxy"
(line 44)

I already tried looking up for a solution on google but I cant find it, this is the script for vmtoon:

# pre-release vmToon v0.7
# used to be known as mfToon
# pre-release because vectrangle is in the opposite timezone of me

# if you're wondering why 0.7 and not 0.6, mftoon 0.6 was never released

function mf_min(string a, string b)
{
return a + " " + b + " < " + a + " " + b + " ?"
}

function mf_max(string a, string b)
{
return a + " " + b + " > " + a + " " + b + " ?"
}

function mf_str_level(string x, int in_low, int in_high, int out_low, int out_high)
{
return mf_max(mf_min(x + " 16 - 255 * 219 / " + string(in_low) + " - " + string(in_high-in_low) + " / " + string(out_high - out_low) + " * " + string(out_low) + " + 219 * 255 / 16 +", "235"), "16")
}

function vmToon(clip input, int "strength", int "luma_cap", int "threshold", int "thinning", \
bool "sharpen", bool "mask", bool "show", string "showclip", int "ssw", int "ssh", int "xstren", \
int "xthresh") {

# vectrangle's stuff

str = String(Default(strength, 48) /128.) # line darkening amount, 0-255
lum = String(Default(luma_cap, 191)) # bright limit for line detection, 0-255 (255 = no limit)
thr = String(Default(threshold, 4)) # threshold to disable slight darkening (of noise) 0-255
thinning = Default(thinning,24) # line thinning amount, 0-255
thn = String(thinning /16.)

# mf's stuff

sharpen = Default(sharpen, true) # sharpening on/off
mask = Default(mask, true) # masking on/off
show = Default(show, false) # show the showclip or the output, true/false
showclip = Default(showclip, "lines") # show the lines or something else
ssw = Default(ssw, 4) # supersample factor horizontally, 0-inf
ssh = Default(ssh, 4) # supersample factor vertically, 0-inf
xstren = Default(xstren, 255) # xsharpening strength, 0-255
xthresh = Default(xthresh, 255) # xsharpening threshold, 0-255

mt_lutxy(input, input. blur(1.0), mf_str_level("x y - abs 128 +", 132, 145, 0, 255), y=3, u=1, v=1).blur(1.0).mt_lut(mf_str_level("x", 0, 64, 0, 255), y=3)

##
edgemask = last

# vectrangle's stuff

exin = input.mt_Expand().mt_Inpand()
diff = mt_lutxy(input, exin, YExpr= "y "+lum+" <y> x y " \
+lum+" < y "+lum+" ? - 0 ? 127 +")
linemask = mt_lutxy(diff.mt_Inpand(), "x 127 - "+thn+" * 255 +") \
.mt_Convolution("1 1 1", "1 1 1", Y=3)
thick = mt_lutxy(input, exin, YExpr= "y "+lum+" <y> x y " \
+lum+" < y "+lum+" ? - 0 ? "+str+" * x +")
thin = mt_lutxy(input.mt_Expand(), diff, YExpr="x y 127 - "+str+" 1 + * +")
(thinning == 0) ? thick : mt_merge(thin, thick, linemask, Y=3, U=1, V=1)
darkened = last

# mf's stuff

MergeChroma(Overlay(input, BlankClip(input, color=$00FF00), mask=edgemask))
lines = last

mt_merge(input, darkened, edgemask, Y=3, U=2, V=2)
masked = last

mask ? masked : mergechroma(darkened, input)
LanczosResize(width*ssw, height*ssh)
XSharpen(xstren, xthresh)
LanczosResize(width/ssw, height/ssh)
sharpened = last

showclip2 = Eval(showclip)
sharpen ? sharpened : mask ? masked : darkened
show ? showclip2 : last

}

Hope that someone had the same issue in the past and remembers how they solved it.

Scintilla · Post by **Scintilla** » Mon Aug 04, 2008 5:22 am

oside wrote:Avisynth Open Failure:
Script Error: there is no function named "mt_lutxy"
(line 44)

In that case, you need the MaskTools 2.0 (MT_MaskTools) DLL. The latest version is <a href="http://manao4.free.fr/masktools-v2.0a35.zip">here</a>.

oside · Post by **oside** » Mon Aug 04, 2008 2:59 pm

Thank you Scintilla! I already extracted the files to the Avisynth plugin folder, and unfortunately gives me the same error. The script that Zarxrax actually seems to work but creates the weird green screen. I'm ready to give up. Is there something else that I can do.
1. What other steps to I need to do with the rar that Scintilla gave me?
2. What do I need to change on Zarxrax script so I can get rid of the green screen effect?

PLEASE HELP!!!!!

Scintilla · Post by **Scintilla** » Mon Aug 04, 2008 4:32 pm

You should only have extracted <b>mt_masktools-26.dll</b>; if you also extracted the "25" DLL, then delete it.

Just to be sure, what version of AVISynth itself are you running on here? The new MaskTools should work on 2.5.6 and up, but probably won't work on 2.5.5.