Archive notice: This article was originally published on May 9, 2013. Links and embedded videos are preserved as part of the historical record.
The ILM team consists of Hao Li, Jihun Yu, Yuting Ye and Chris Bregler, who will present a real-time-capable, calibration-free facial performance capture framework based on video and depth input at this year’s Siggraph.
For this framework, an adaptive PCA model was developed using Shape Correctives that adjust the actor’s facial expressions, simply on the fly, through incremental PCA-based learning.
Because the model increasingly improves itself during the performance, no exra caputre or training session is required to build the model.
Here is roughly what the whole thing will look like:
