Use a ready-made set of animated mouth movements to match the speech in any language

Around the world, each spoken language has distinct units of sound called Phonemes . While the sound is produced using lungs, throat and mouth, the viewers can see only the movements of the lips, tip of the tongue and the lower jaw (including the teeth).

Thus, an animator has to animate these three parts to lip-synch the character's face with his speech.

Further, in a given language, multiple phonemes share a common facial pose.

For example, in English, p, b, and m share the same facial pose. So if the character says "pace" or "mace" or "base", it uses the same common facial pose.

Similarly, the same facial pose is used to say "thin", "tin" and "din".

Given this, we should have a system to convert a text into a sequence of facial poses.

The idea is just like Text-To-Speech (TTS), which we are familiar with. Here, the proposal is to convert the text to phonemes first, and then to standardized facial movements.

In addition, the animator should be able to customize the standard facial movement to represent some typical facial movements, or add a "bias" to represent the person's personality (cheerful, heavy-hearted, stern), or just facial asymmetry.

For example, Drew Barrymore, Sylvester Stallone and Alicia Silverstone move their jaw/lips to one side. When Arnold Schwarzenegger speaks, his lower jaw opens wider. In the Godfather series, Al Pacino has a prominent jaw and a downturned mouth. These nuances can be reproduced accurately by using a bias.