Disney’s Research Studios have published a research paper about a new artificial intelligence (AI) system for digitally re-aging actors. The system is called the Face Re-Aging Network, or FRAN for short.
What is re-aging?
Filmmakers often need to make actors look older or younger to show the passage of time in a movie. Traditionally this would have been done with practical makeup effects applied to the actors’ faces. This has now been largely replaced by digital visual effects. However, the process is complicated and takes a long time, meaning it’s quite expensive. Martin Scorsese’s film “The Irishman” (2019) required two years of post-production and a budget possibly as high as $250 million to make the cast look younger.
How is FRAN different?
There are two current methods for digital re-aging. One involves scanning the actors’ faces and creating detailed 3D models. The other involves artists literally digitally repainting the actors’ faces on each frame of the movie. However, Disney’s FRAN uses an AI neural network based on a system called U-Net, which was originally designed for medical purposes. FRAN analyses the existing video and applies the re-aging process, working as fast as five seconds per frame.
How does FRAN know what to do?
Disney had a major hurdle to overcome so it could train FRAN on how human faces age. They needed photos of people at different life stages to show their aging. However, collecting these photos with the same poses and lighting is nearly impossible. Disney solved this challenge by using photorealistic computer-generated images of faces which were then aged using existing software. By creating a series of still images showing a face at different ages, from 20 to 80 years old, Disney was able to teach FRAN how humans age.
How good is FRAN?
Disney’s research paper and the associated videos include examples of subjects that have been aged by FRAN. The results look very good, with the system dealing with changes in the position of the head and the lighting of the scene. There are also examples showing that FRAN can provide realistic results even when there is motion blur in the frame. Although the system isn’t perfect, it can be adjusted by an artist to apply increased aging effects to certain parts of the face, such as around the eyes, where needed.
Disney’s research paper promises an exciting future for the digital re-aging of actors in movies. By saving time and simplifying the re-aging process, FRAN will save a lot of money in movie production. It’s still a system under development, though, and the paper acknowledges that further work needs to be done. For example, FRAN does not turn an actor’s hair grey as it ages them, so this would still need to be done with practical makeup effects. However, Disney is continuing to improve the capabilities of FRAN, so watch this space. We will keep you updated with any future developments.