Faceware Analyzer

Written by: Orion Tighe

Helpful Resources

http://support.facewaretech.com/home

http://support.facewaretech.com/analyzer

http://support.facewaretech.com/retargeter

Faceware Youtube Channel

• Get video from Facial Videos folder in drive
  • Check video, make sure it’s full length (contains the whole section of dialogue), or that dialogue is present across collective clips
    • When recording the face videos, the GoPro would split the recording at ~8 minutes, making a few different clips for one piece of dialogue
  • Try to choose video with most accurate mouth movements - eyes are taken care of by Salsa in Unity, and brows are straightforward
  • Download video and move to documents Facial Videos folder
• Open selected video in Faceware Analyzer
  • Analyzer does better with smaller segments
    • When opening video, select the timeframe in seconds to work with - cut out excess movements from the beginning before speaking occurs, and work in roughly 2-3 minute segments
    • Make sure to mark down where you split! Animation checklist doc, desktop sticky note, txt file - just write it down somewhere
    • Each time you make a new faceware tracking file, overlap with the previous clip by a second or two
  • When making the analyzer clip, you will:
    • Select the video clip (from the facial videos folder)
    • Name the job
      • I named the monologues as MonoX, where X is the clip number; and the dialogues as DialogueX, where X is the number of times they’ve spoken (first time they speak is Dialogue1, second time is Dialogue2, etc. Feel free to name them however you’d like, as long as you’re keeping them organized
    • Assign a job location
      • By default, Faceware will try to place the job in the video folder. Move it to a better place: Documents/Great Experiment/Faceware/ActorName/JobName, i.e. Wilson/Dialogue1. You’ll have to manually create a new folder for the job, then select that folder as the job location where FW will generate it’s files
    • Trim the job to the desired length in seconds
• Train/track the video clip in Analyzer:
  • Make sure you’ve reviewed the tutorials and guides online, and reference them regularly http://support.facewaretech.com/home 
  • Do not work on the Face Group! That is for ‘review’, but realistically you’ll never even use it
  • Complete each layer one at a time
  • For the first time you track and actor, make three keyframes across the timeline in the layer: generally you’ll have one near the beginning, and then two more in extreme poses
  • Once you have three keyframes, click Train; after it’s finished, click Track; then, scrub through the timeline to look for places where the analyzer loses tracking or is inaccurate and make new keyframes in the worst spots
    • Important: any time you make a keyframe, align ALL nodes with the correct spot! Do not adjust only a few nodes, makes sure they are all moved to the correct position
    • When moving nodes, you can enable Intelligent Drag in the toolbar, or hold down Shift when it’s disabled to use it. This will move all unaligned nodes to logical spots
    • When moving nodes, make sure you are putting the correct node in the correct spot! It can be very easy to accidentally flip the left and right sides of a pupil, or switch the inner and outer mouth nodes. Doing so will generally cause major glitches and bugs in the tracking; if you find that a training model is not working as it should, check if everything is in the correct spots
    • Personally, I’ve found more success manually scrubbing frame by frame (holding the left or right arrow key) than using the play button, since it plays slightly slower and smoother
    • When tracking the brows, place the inner landmark on the upper corner of the brow and the outer landmark on the edge
    • Most of these details are covered in the Faceware guide and video tutorials, as are the hotkeys and shortcuts. Watch them!!
  • If this is not the first time training an actor, you will likely already have a usable training model. Before you make keyframes, import the training model for the actor under File. Then, you will only need to make one keyframe before training and tracking the first time.
• Finish tracking the clip:
  • The first time you train an actor, you will not have a training model yet. Go to the actor’s folder under the Faceware folder, and make a new folder named [Actor] Model. In Faceware, go to File>Export Tracking Model and export it to the folder you just created. From then on, any time you do a job with the actor, you can import that model to make your life a lot easier.
    You also need a Neutral Frame, as it can make things a bit easier. Find a spot in the timeline where the actor’s face is as neutral as it can get, then go to File>Export neutral frame. The frame you have selected will need all nodes aligned; this can be done easily by selecting landmarks you want to mark and hitting Ctrl+T. Simply save the neutral frame in the actor’s faceware folder.
  • Any other time you finish working on an actor, you’ll export the training model again. Select Export Training Model, and choose the original folder for the model; it will overwrite the old model with the newer, better model generated with the current job.
  • Finally, once the job is accurate enough and you’ve exported the new training model, you’ll click Parameterize in the top right corner. This will take some time, and generate a performance file that will be used in the Retargeter in Maya.