Masking/Compositing in AE

ssj4lonewolf · Post by **ssj4lonewolf** » Fri Aug 26, 2005 3:14 pm

ok ill give it a shot, cuz ap is confusing as hell right now

Org Profile · Post by **Vlad G Pohnert** » Tue Sep 20, 2005 1:47 am

Didn't see this thread yet...

I've now started to go beyond just masking in After Effects. The total amount of combinations you can do with the program is incredible and can't be done in video editing programs (premiere) as mentioned...

It's also been my experience with complex masking that using Bezier Curves can also be a pain in a some cases... I personally use point to point with complex character outlines as it much eaiser to control color bleeding edging and I find it give the more realistic feel. I use a ton of points (yes it's insane, but the results can be staggering) and more them in a group as a lot of stuff in anime characters in some ways stays the same.

Another good trick is if say you want to have the hair of a character wave back and forth in the wind in your cutout, you can use a short sequence of movement and just copy it in the total number of cycles needed. In a lot of anime after a while you will notice that some scenes us this technique to save on production costs in the first place...

Best thing I find is to experiment with both curves and points to see what works best for you. There is no absolute correct way and it's the final results that are important.

Vlad

JudgeHolden · Post by **JudgeHolden** » Wed Oct 12, 2005 12:56 pm

Ha! This will work well in Motion 2 and Final Cut with the new masking plugin. Thanks for the posts!

JH

Gaarasan · Post by **Gaarasan** » Sun Nov 06, 2005 8:10 pm

Can anyone please provide something like a video tutorial or a tutorial with pics in doing this in after effects? Because I just started using after effects and it's so not user friendly.

Ratix · Post by **Ratix** » Mon Nov 07, 2005 12:04 am

TaranT wrote:BTW, the crude way to mask the characters would be to export the image sequence and edit each frame in an image editor. Guaranteed to work, but don't use VideoFactory. It's not fun with large numbers of single frame pics.

I'm pretty sure you can use AVISynth to recombine a sequence of images back into a video stream.

Kira_Douji · Post by **Kira_Douji** » Tue Nov 22, 2005 7:46 pm

Ashyukun wrote:

Wow. That's quite a write up, E-ko. You should (if you haven't already) write that up as a guide.

When I do rotoscoping of characters into other series and such (WLtFO, Samurai Windu) I tend to do it entirely in After Effects using its internal masking capabilities. This has both its advantages and its disadvantages. Disclaimer: I'm writing this up from memory, I don't have a computer with AE in front of me, so hopefully I won't screw things up too badly. Also note the keys I talk about are the PC shortcut keys... I imagine they're similiar on a Mac, but I don't know for sure ... I'd also advise reading through all of this before trying it...

So yeah... I think you have an older version of After Effects? I've been trying to step through this and you lose me at about "changing the color of the mask". >>; What you say should be there (the menu for changing the colors) simply isn't. I'm running After Effects 6.5.1 Pro... is your version different? If so, do you know how to continue on after you've made the mask in 6.5.1 pro?

- Kira

Kira_Douji · Post by **Kira_Douji** » Tue Nov 22, 2005 8:40 pm

Well, nevermind, I figured it out. Thanks for the walk through explaining the basic concepts, though, it certainly helped

Gepetto · Post by **Gepetto** » Thu Dec 29, 2005 1:51 pm

You can call me thick or stupid, but I didn't get E-Ko's explanation... is there a guide or something on doing this without Photoshop?

Perfect-Blue · Post by **Perfect-Blue** » Fri Dec 30, 2005 5:20 pm

forget about photoshop - its waste of time,and the result is the same ,if u make masking only with AE.But is more faster2.

DriftRoot · Post by **DriftRoot** » Sun Jan 08, 2006 11:58 am

Don't eat me, please help me. You have to understand that this masking stuff is going to doom my AMV to be unless I get it to work.

I need to mask out the background of a certain scene so that only the character is "visible." Simple, huh? He will be placed against a new background, eventually. I slapped a mask on him in AE, but it took forever using that dratted pen tool, so I figured I'd export the frame to PS, create a mask a la alpha channel goodness, and import it back into AE.

Maybe this is where I'm going wrong. Maybe I can't do this at all (the alpha channel masking in PS for use in AE). Maybe I need to export every frame to PS, cut out what I don't want, and import it. But this just doesn't seem logical. If I've got to mask out the background of a static character whose eyes/mouth is moving, this just won't work and this is what masks are designed to handle.

The problem is that I cannot create an alpha channel mask in PS. I cannot get my alpha channels to stick. No, not even stick, apply themselves! What's worse, it used to work for me. Once upon a time, using E-ko's method, I actually created an alpha channel mask that worked like it was supposed to in Premier. Now, suddenly, for no apparent reason, It. Does. Not. Work. For. Me. I've referred to other, quite similar methods for mask creation, and THEY do not work for me.

I think something's bonkers with my PS program. Up until the point at which I hit "apply layer mask" everything looks decent (alpha channel exists, with correct transparency). However, applying the mask to my target layer deletes the entire contents of that layer, and of course takes my carefully constructed alpha channel mask with it! PS never did this to me before...

Maybe my ideas about alpha channel masks are messed up...maybe that's the problem.

Maybe what I thought I did when it *worked* was something else completely? I have no idea. But I'd love to figure out what's wrong.

p.s. I can cheat a little using alternative blending modes in Premiere/AE for B&W images used as masks, but it's not ideal. I also can cheat a bit using track mattes, but again, it's not ideal. Alpha masks are such a simple thing to create, and yet I can't seem to do it. Any advice (including reinstalling my PS program, which is what will happen if it doesn't start cooperating) is much appreciated!