Hedra can give voice to AI characters - this new model is a big step forward!

Hedra can give voice to AI characters - this new model is a big step forward!

Hedra, an AI character generation platform, has announced a new version of its model.

It also has a new “stylize” feature that allows users to dress their characters in different outfits, add backgrounds, and change their faces to Lego characters or cartoons.

Characters are Hedra's new family of foundational models designed to create more consistent and realistic humans through AI video, giving creators more control over the final output. Initially, the focus is on lip-sync and head animation. Version 1.5 adds that functionality.

Hedra offers a generous free plan for people to try it out, and I found it works with any human image, including photos, drawings, and even action figures. I was able to animate both the head and face fairly accurately to the words used in the voice track.

I created five characters. First, I used Flux.1 running locally on my laptop to display the image prompts, then used ElevenLabs to create the voices for each character. Finally, the voices and images were used to create a video using Hedra.

Each image was of a person looking straight at the “camera” and each was different enough to be useful for testing. We then ran stylize on two of them to see the very different facial expressions and the subtle changes in costume and background. Below each description is the full prompt I gave Flux.1 to create each character.

First, we have a middle-aged doctor in a white coat. Then I styled him in a bright, exaggerated costume filled with colorful backgrounds and multicolored splashes.

He did not do a bad job. There are artifacts around the mouth and the head movement is a bit exaggerated, but it reflects the tone of the sound well. Voice matching was not AI, but ElevenLabs found the right voice.

Next is the builder character. This is not a test of the AI image generator, but of Hedra's lip sync and head movement animation capabilities.

The head movements are more natural here, with minimal artifacts, but the blinking is unnatural. Still, it is a significant improvement over Hedra Character-1 and other AI lip-sync tools.

The barista character had the most natural blinking of any of my tests. She also had slightly exaggerated lip movements, but overall a good rendering. She maintains the familiarity of the first image and voice.

I am not too happy with how this image turned out. It looks more like a bad staff proof photo than a video of someone talking to the camera, but the realism of the image seems to help Hedra.

Finally, an older face. I had the audio create pauses in the middle to accurately reflect the animation of Hedra taking a breath/gathering her thoughts.

As with all tests, the mouth and head movements were exaggerated compared to reality, but a great improvement.

I was already a fan of the way the lip movements were animated in v1, but Hedra in 1.5 takes it to another level by adding more natural head movements and facial expressions.

Stylise is another powerful addition that gives us some insight into what we might see in the final, fully controllable AI video model that Hedra is developing.

This feature makes it easy to adapt any element of a character photo, changing its appearance or even just the clothes it is wearing in the image.

All we need now is a widescreen and portrait option, so we can better integrate into projects made with a growing number of AI video products, including Runway and Kling.

Categories