Uses audio analyzis of user speech to apply lip synchronization without the use of face tracking hardware. The result of the lip synchronization provides weights for visemes. These weights are used as vertex position set weights (blend shapes or morphs in other applications) to perform mouth animation. Each viseme is assigned a vertex position set.

Three operating modes are supported. The first applicable mode is used:

1. Full Viseme: At least one viseme of the full viseme group is defined. The lip synchronizer provides the weights for each viseme and applies it to the corresponding vertex position set.
2. Viseme subset: At least one viseme of the viseme subset group is defined. This provides a subset of mouth poses consisting of 4 basic shapes. This is sufficient for many situations since games tend to have voice acting in one language but the user often reads the closed captions in a different language.
3. Open mouth move: An animation move is blended based on the mouth openness value of the lip synchronizer. This is fast to do since it requires only a single animation move. This is best used for characters with elongated snouts or mouth shapes where complex mouth animations are possible to do in a reasonable way.
4. Jaw bone: A jaw bone is rotated based on the the mouth openness value of the lip synchronizer. This is the fastest solution not requiring any upfront modification of the character. It has the same use case as Open mouth move.

The visemes line up with the visemes defined by VRChat and similar applications.

Name of motion transfer to identify it in the list. Name is not required to be unique.

Configures how the mouth is driven if no viseme vertex position sets are assigned. The open mouth move and jaw bone parameters are exclusive. The open mouth move takes precedence over the jaw bone if both are set.

Animation move from the character animation file to blend for opening the mouth. The blend factor of the animation move is driven by the detected mouth openness ranging from fully closed to fully open.

Bone to rotate as an alternative to an open mouth animation move for driving jaw flap. Click on the label or the combo box to activate bone selection mode. Click in the preview window on a bone to select it. The selected bone is marked in red.

Rotation axis of the fully open mouth along the X axis (red axis). Click on the label or the text widgets to activate coordinate system mode. A coordinate system gizmo is shown in the preview window at the position of the jaw bone. Rotate the coordinate system until the X axis (red arrow) aligns with the jaw rotation axis.

Rotation angle in degrees of the fully open jaw bone.

Assigns vertex position sets for the basic subset of four visemes. These four visemes are sufficient for basic lip sync animation when a complete set of mouth shapes is not available in the character model. Each combo box shows the vertex position sets defined in the character model. Assign the vertex position set matching the mouth shape for the respective viseme.

Neutral mouth position used when no sound is produced. Assign the vertex position set representing the default closed or resting mouth shape. If no All Visemes viseme is set this is also used for closed mouth sounds like m or p.

Open mouth shape. Used for back open vowels, for example a in “father” or o in “hot”.

Rounded open mouth shape. Used for back rounded vowels, for example o in “go” or aw in “saw”.

Slightly open mouth shape. Used for mid front vowels, for example e in “red” or eh in “bed”.

Assigns vertex position sets for all visemes not covered by the simple subset. Using the full set of visemes produces more nuanced and natural looking lip sync animation. The character model needs vertex position sets matching each assigned viseme. Each combo box shows the vertex position sets defined in the character model. Assign the vertex position set matching the mouth shape for the respective viseme. Visemes left unassigned are ignored.

Narrow horizontal mouth shape. Used for close front vowels, for example i in “bit” or ee in “bee”.

Rounded and nearly closed mouth shape. Used for close back rounded vowels, for example u in “you” or oo in “too”.

Closed lips shape. Used for bilabial consonants, for example p, b and m.

Lower lip touching upper teeth. Used for labiodental consonants, for example f and v.

Tongue near or between the teeth. Used for dental consonants, for example th in “think” and th in “the”.

Tongue tip raised near the upper teeth and alveolar ridge. Used for alveolar consonants, for example d, t and l.

Back of tongue raised toward the velum. Used for velar consonants, for example k, g and ng.

Rounded mouth with tongue near the hard palate. Used for palato-alveolar consonants, for example ch, j and sh.

Teeth nearly closed with air passing between them. Used for alveolar fricatives, for example s and z.

Near-closed mouth with air flowing through the nose. Used for nasal consonants, for example n and ng.

Slightly open and rounded mouth. Used for rhotic consonants, for example r.