How is lip syncing done?

Shadow Wolf · Post by **Shadow Wolf** » Tue Apr 27, 2004 9:27 pm

How does a person go about attempting to do lip syncing? Is that a plugin with Premiere, done with effects and timing, After Effects or something entirely different?

Scintilla · Post by **Scintilla** » Tue Apr 27, 2004 9:46 pm

http://www.animemusicvideos.org/guides/ ... psync.html

Kalium · Post by **Kalium** » Tue Apr 27, 2004 9:56 pm

One thing that's not mentioned there that I find useful is the waveform. If you're doing a trailer or something where the audio is nothing but the voice, then the waveform is a big help. Premiere Pro shows the waveform on the audio track, so that's very handy.

Scintilla · Post by **Scintilla** » Tue Apr 27, 2004 10:53 pm

Kalium wrote:One thing that's not mentioned there that I find useful is the waveform. If you're doing a trailer or something where the audio is nothing but the voice, then the waveform is a big help. Premiere Pro shows the waveform on the audio track, so that's very handy.

Ehhh, I find the waveform more useful for synching beats; for lip synching I just scrub the audio.

fyrtenheimer · Post by **fyrtenheimer** » Wed Apr 28, 2004 9:15 am

Kalium wrote:One thing that's not mentioned there that I find useful is the waveform. If you're doing a trailer or something where the audio is nothing but the voice, then the waveform is a big help. Premiere Pro shows the waveform on the audio track, so that's very handy.

as far as I know, 6-6.5 all do the same thing

Hitori · Post by **Hitori** » Wed Apr 28, 2004 9:58 am

fyrtenheimer wrote:
Kalium wrote:One thing that's not mentioned there that I find useful is the waveform. If you're doing a trailer or something where the audio is nothing but the voice, then the waveform is a big help. Premiere Pro shows the waveform on the audio track, so that's very handy.
as far as I know, 6-6.5 all do the same thing

It does...

I should post a tutorial on lip syncing on my site!!....
Well, the way I've been doing lip syncing... It's different from what's shown in the guides. More like an advanced way to keep it seamless and combines almost everything from the lip syncing guide...

(The tutorial for my site will include examples from my next trailer parody.... Almost done! A lot of work went into this one...)

godix · Post by **godix** » Wed Apr 28, 2004 7:48 pm

I've done videos where I said 'Eh, the mouth is moving at the right time, good enough' and I've done videos where I'd sit there and say the word out loud several times just to make sure I got the mouth movements right then I'd spend forever trying to get the damned thing frame perfect. The difference between these two methods is blatently obvious. This isn't just me being picky about my own videos either, I've gotten ops that mention the lip syncing wasn't that good on the 'flapping lips' method and the ones I've gotten ops saying it's some of the best lipsyncing the person has seen on the ones I spent forever with. Basically my point is that if you want good lipsyncing then don't get lazy, spend the time it takes to get the mouth movements right.

Of course this depends on the source. Someone speaking in a well animated anime will have 3 or 4 different positions drawn for their mouth. If it's one of those animes that's not all that well animated there will just be two, wide open and closed. With that type of footage I doubt you could make it look good no matter how much time you spent with it.

madbunny · Post by **madbunny** » Fri Apr 30, 2004 2:40 am

Scintilla wrote: Ehhh, I find the waveform more useful for synching beats; for lip synching I just scrub the audio.

x2 Scintilla
x2 Godix
You could always run the audio through a bandpass filter or two to isolate the audio as much as you can for a visual reference though.
There is really no substitute for nitpicky hard frustrating detail when it comes to lip sync.

Wheee_It's_Me! · Post by **Wheee_It's_Me!** » Fri Jun 04, 2004 12:27 pm

The way I do lip synching is pretty complex. It involves a combination of Premiere, Animation Shop and Gold Wave and occasionally ACDSee in conjunction with a scroll mouse (to flip between frames to check the smoothness of the animation). THe later is only important though if you're doing reanimation, not if you're just doing static frame lip synching.

At the basic level you're going to need three mouth frames, one closed, one half way open, and one wide open. You should create these frames in like Paint Shop and rename them as such:

01 - closed.bmp
01 - half.bmp
01 - wide.bmp

The next thing you need to do is build a word base (again, this is for doing static frame lip synching only). To do this you first need to analyze the lyrics. Like let's take the word "before". Say it out loud a few time and really enunciate your lip movements. You should find that your mouth starts out halfway open and then goes to wide open. Now you need to REALLY pay attention to the lyrics in the song. Do they say the word fast or slow?

If they say it slow, open up animation shop, run the wizard and choose the following setup:

01 - closed.bmp
01 - half.bmp
01 - closed.bmp
01 - half.bmp
01 - wide.bmp
01 - half.bmp

if the word is said very quickly, choose the following setup:

01 - closed.bmp
01 - half.bmp
01 - wide.bmp
01 - half.bmp

if the word is said REALLY fast, you might choose the following:

01 - closed.bmp
01 - half.bmp
01 - wide.bmp

Again, you have to kinda give it your best estimation depending on the lyrics. Now comes the tricky part. Open up the audio file in Goldwave, go into the setup and make sure that looping play is set for like a 1,000 repetitions. Now isolate the lyric where he says "before" and make sure it's highlighted and NOTHING else, you want just that one word. Now click on repetitious play.

Now I'm going to tell you, this can REALLY freak you out if you're not used to it, in fact I'm sure for some people it just drives them up the wall. If that's the case the only real recommendation I have is to try and do more than one word at a time, it might be a lil harder to put together, but it won't get on your nerves as bad.

Okay, the word "before" is now repeating incessantly in your ears. Go back to animation shop. Now you need to alter the frame rates, this is the most tricky part and it requires more artistic skill than pin point instructions. I usually like to start out with the first closed frame at 10 and then go from there. The trick is click on view and then click on animation at the EXACT point at which the clip starts to repeat, again if you're not used to it, this might be hard to do.

Now if you did that one of three things will happen.

1. it all matches up perfect (this usually doesn't happen)
2. it doesn't match up AT ALL
3. at first it looks like it matches up, but after 3 or 4 repetitions you find that it's getting farther and farther off.

Okay, scenario 1, save the thing as an avi and call it: 01 - before.avi

Scenario 2, chances are you misjudged the lyric length, you might have to eliminate some of the frames.

Scenario 3, you're REAL close, try altering the frame rates of some of the frames by about 5 to 10 and retest it again and again until it repeats perfectly. Once it does, save it as I mentioned before.

YAY, you finished one word! ^_^

Now you get to repeat the process over and over and over and over and over again! And then once you have all your lil bits, you need to slap them into Premiere and line them up properly (usually this step goes pretty easy). Occasionally you have to make a minor adjustments by adding a closed mouth sequence between clips. To test the smoothness, zoom into a level of about 8 frames and use the arrow keys to move through the animation. Fast enough to make sure it lines up, but not so fast that you're going to fudge and not so slow that you're not going to be able to tell when something is said clearly. Again, it takes some practice.

And THAT my friends is the right way to do lip synching...albiet tedious and incredibly time consuming, the results are quite nice.

So...does anyone want to know how to do head motion lip synching now?

PS - there are some other methods too, like downloading the actual music video for a song and then analyzing it frame by frame...but that only works if you can clearly see the singer mouthing the lyrics you want to synch.

There's also another method by using a capture program like Windows Media Encoder in conjunction with the visualizations in Windows Media Player to capture the wave form as a video file and then construct it that way (replacing dead zones with closed mouths, half spikes with halfway open mouths and full spikes with wide open mouths). That method however is dependant on how well you can isolate the lyrics from the background sound. Really if you want to do it that way I'd almost suggest donning head phones and a mic and then singing the lyrics yourself as you listen to the music, that way you can get a clear cut of the lyrics with no music...however it's also slightly dependant on your kareoke ability. *shrugs*

Corran · Post by **Corran** » Fri Jun 04, 2004 2:59 pm

Uh... I stick the video I want to lip sync on the timeline, find particular mouth shapes that I want, cut the video up at the frame level and then I insert/copy/paste the frames so that they make the right shapes at the right time. So far it works perfectly but it is rather tedious to work at a frame by frame level.