Trying LivePortrait - a fun way to animate headshots

Trying LivePortrait - a fun way to animate headshots

Come on, let's admit it. You probably have hundreds, if not thousands, of family portraits sitting on disks or devices in your home. Now there is a new online platform called LivePortrait.

This open source project from Kuaishou Technology and Fudan University in China uses specially trained models that can convert animation from video or webcam and transfer motion to images and photos.

This is very clever and an example of how widespread and rapid this type of AI technology has become.

What sets LivePortrait apart from other portrait animators is the amount of customization that can be used on the image to create the final effect.

I tried several experiments using standard demo videos and images, as well as my own images. The first attempt was to animate our beloved old Mona Lisa, which worked better than expected.

Lisa returned a stunning facial expression, triggered by the sample video portrait on the site. However, when she tried it on her own uploaded video, the results were less expressive.

Obviously, expressive movement of the eyes and lips is necessary to get the most out of this platform, but it is still quite good. My Lucy character was just talking to the camera, so the girl with the pearl earring looked more like a mumble than anything else.

However, the ability to custom set the degree of lip and eye movement, which they call "retargeting," is very cool and a great demonstration of the technology behind what looks simple.

The platform uses about 69 million high-resolution video frames as training data and very clever keypointing techniques to achieve a facial shape that makes the end result look very good.

Categories