TalkToMe 1.0
lips syncing technology for everyone
Yellow House Studios
Introduction
TalkToMe is a small SDK that allow you to automatically lips-sync your characters with any audio sample. It is very simple to use and fast to implement in your application.
Licencing
Talk-To-Me was designed with the indie-dev in mind so the price is very low for this type of technology. There is two different licence. The first one will cost you 150$ and nothing else is requiered from you. The second one will only cost you 35$ but you will have to include a link on your website linking to www.ovogame.com, also you should credit us in your credits (again www.ovogame.com) plus a free copy of your game will be welcome.
if you wish to purchase the 150$ licence click here.
if you wish to purchase the 35$ licence click here.
SDK
You can download the SDK here. It is composed of a main application: TalkToMe.exe. This application allows you to visualize the lips syncing technology against any wav file. The wav file format must be PCM, 48000kHz, 16 bit mono. This application also allows you to create the ttm files (ttm standing for TalkToMe). These ttm files contain the data for the lips syncing. They are the only files you will need to load in your game. The SDK also contains few .cpp and .h in the Src folder. Include these 5 files in your project (TalkToMe.h, TalkToMe.cpp, Compression.h, Compression.cpp, Header.h). These files should be self contained. The only file that needs to interact with your project is TalkToMe.h.
To create a ttm file, you need to load the raw data by your own means and call CTalkToMe::Create(ptr_raw_data). This function returns a pointer to an instance of a CTalkToMe class. You access the lips syncing data by calling the function GetValues(float *array, float pos). The lips syncing data is made of 3 floats. These floats are the coefficients (percentage) of the 3 following mouth position:
Depending on your visualization, 3D (left) or sprite (right), you will have to do different process. For the 3D (red lips), we are using a morph target technology but you can also use a system based on bone technology. You will need to be able to mix 4 different shape or animations frames. The 3 preceding shapes plus the neutral shape (mouth closed).
If you are using a sprite based engine, then it is easier. You will have to analyze the 3 different floats and snap to the closest frame. Check the FAQ to find the algorithm we used for our sprite base animation (right).
The second parameter of the GetValues function is the position of your audio sample. This position is a float between zero and one. The way you get this position will depend on the audio engine you are using. If your audio engine doesn't support this feature, read the documentation again, it is really unlikely that it doesn't ;-). For our example, we are using FMOD, this is how we get the position of the audio between zero and one:
To resume this is how you src code should look like:
Special Thanks
Rifts Unlimited (MC Rifts & Mr Scotland)
FAQ
Where can I find the SDK?
You can download it here.
The save option doesn't work. What is going on?
You need to licence the SDK to get the complete version. For each wav file provided with the SDK, you will find a ttm file (in the TTMs folder). Use these files to test the SDK before buying.
How can I change the format of my wav file if they don't match the required format?
Use the "Sound Recorder" application from windows. Click on the start button then select All programs.You will find this application in the Accessories\Entertainment folder. Load your sample and select "Save As". At the bottom of the new window you will find a change button. Click it and set the parameters as follow.
The format for the wav file is very inefficient (uncompress). Do I need to use this wav format in my game?
No, not at all. You need to use the previous wav file format only when you extract the lips syncing data. This is done as an offline process. Your game will only need to read the ttm file (lips syncing data). The way you play your audio file in your game is up to you. You don't even need to use the wav file format if you have a different system in your game.
Can you tell us how to analyze the 3 floats to decide witch sprite to use if we are using a sprite system?
The following code will set the variable win with the index of the texture to use. Assuming the order is the following:
I have a problem and this FAQ is unhelpfull, what should I do?
You can contact us at ovo@ovogame.com but don't forget to mention in details what is your problem.
I have a webcam, is there anything you can do for me?
Yes, why don't you test our Webcam based game OvO ;-).