Faceware Analyzer

Written by: Orion Tighe

Helpful Resources

Check video, make sure it’s full length (contains the whole section of dialogue), or that dialogue is present across collective clips

When recording the face videos, the GoPro would split the recording at ~8 minutes, making a few different clips for one piece of dialogue

Try to choose video with most accurate mouth movements - eyes are taken care of by Salsa in Unity, and brows are straightforward

When opening video, select the timeframe in seconds to work with - cut out excess movements from the beginning before speaking occurs, and work in roughly 2-3 minute segments

Make sure to mark down where you split! Animation checklist doc, desktop sticky note, txt file - just write it down somewhere

Each time you make a new faceware tracking file, overlap with the previous clip by a second or two

I named the monologues as MonoX, where X is the clip number; and the dialogues as DialogueX, where X is the number of times they’ve spoken (first time they speak is Dialogue1, second time is Dialogue2, etc. Feel free to name them however you’d like, as long as you’re keeping them organized

By default, Faceware will try to place the job in the video folder. Move it to a better place: Documents/Great Experiment/Faceware/ActorName/JobName, i.e. Wilson/Dialogue1. You’ll have to manually create a new folder for the job, then select that folder as the job location where FW will generate it’s files

Make sure you’ve reviewed the tutorials and guides online, and reference them regularly http://support.facewaretech.com/home

Do not work on the Face Group! That is for ‘review’, but realistically you’ll never even use it

For the first time you track and actor, make three keyframes across the timeline in the layer: generally you’ll have one near the beginning, and then two more in extreme poses

Once you have three keyframes, click Train; after it’s finished, click Track; then, scrub through the timeline to look for places where the analyzer loses tracking or is inaccurate and make new keyframes in the worst spots

Important: any time you make a keyframe, align ALL nodes with the correct spot! Do not adjust only a few nodes, makes sure they are all moved to the correct position

When moving nodes, you can enable Intelligent Drag in the toolbar, or hold down Shift when it’s disabled to use it. This will move all unaligned nodes to logical spots

When moving nodes, make sure you are putting the correct node in the correct spot! It can be very easy to accidentally flip the left and right sides of a pupil, or switch the inner and outer mouth nodes. Doing so will generally cause major glitches and bugs in the tracking; if you find that a training model is not working as it should, check if everything is in the correct spots

Personally, I’ve found more success manually scrubbing frame by frame (holding the left or right arrow key) than using the play button, since it plays slightly slower and smoother

When tracking the brows, place the inner landmark on the upper corner of the brow and the outer landmark on the edge

Most of these details are covered in the Faceware guide and video tutorials, as are the hotkeys and shortcuts. Watch them!!

If this is not the first time training an actor, you will likely already have a usable training model. Before you make keyframes, import the training model for the actor under File. Then, you will only need to make one keyframe before training and tracking the first time.

The first time you train an actor, you will not have a training model yet. Go to the actor’s folder under the Faceware folder, and make a new folder named [Actor] Model. In Faceware, go to File>Export Tracking Model and export it to the folder you just created. From then on, any time you do a job with the actor, you can import that model to make your life a lot easier.
You also need a Neutral Frame, as it can make things a bit easier. Find a spot in the timeline where the actor’s face is as neutral as it can get, then go to File>Export neutral frame. The frame you have selected will need all nodes aligned; this can be done easily by selecting landmarks you want to mark and hitting Ctrl+T. Simply save the neutral frame in the actor’s faceware folder.

Any other time you finish working on an actor, you’ll export the training model again. Select Export Training Model, and choose the original folder for the model; it will overwrite the old model with the newer, better model generated with the current job.

Finally, once the job is accurate enough and you’ve exported the new training model, you’ll click Parameterize in the top right corner. This will take some time, and generate a performance file that will be used in the Retargeter in Maya.