OmniHuman 2.0

OmniHuman 2.0: Audio-Driven Talking and Singing Avatars (Image to Video)

What is OmniHuman 2.0?

OmniHuman 2.0 is ByteDance's audio-driven avatar model. Give it one still portrait and an audio track — speech or song — plus an optional text prompt, and it generates a video of that person performing your recording. Unlike text-to-video, the performance is anchored to your exact audio, so the mouth, expression, and gestures follow what is actually said or sung. Because it animates any single image rather than a pre-trained digital twin, it is fast for new characters and works on stylized subjects, not just photoreal faces.

Key Features

• •Tight lip-sync for both speech and singing from one image plus one audio file.
• •Expressive, meaning-aware performance: facial emotion and upper-body gestures track the content and prosody of the audio, not just its rhythm.
• •Prompt-driven cinematic camera movement — pans, push-ins, and orbits while the character performs.
• •Optional text prompt for scene, motion, and camera in English, Chinese, Japanese, Korean, Spanish, and Indonesian.
• •Long audio support: 4–32s on /v1, up to 112s on /v2; the video is exactly as long as the driving audio.

Best Use Cases

Turn one photo and a voiceover into a talking head without a shoot: spokesperson clips, product explainers, and course narration. It suits AI filmmakers patching in a single line of dialogue or a close-up performance beat, singers and lyric videos built from a portrait, virtual influencers and recurring brand characters, and multilingual localizations of the same script. In testing, a portrait plus a short recording reliably produced lip-synced speech, a believable singing performance, and a prompt-controlled camera move.

Prompt Tips and Output Quality

Use a clear, front-facing portrait with a closed mouth for the cleanest sync. Keep the prompt about scene, mood, and camera — never the words, which come from your audio. Name a camera move (orbit, push-in, pan) to get it. For singing, feed a vocal-forward track; instrumental-only audio produces little mouth motion. Tested output showed tight lip-sync, natural expression shifts, and hands rising into frame, with no gross teeth or hair artifacts.

Using OmniHuman 2.0

OmniHuman 2.0 animates one portrait with an audio track into a talking or singing video. Provide an image, an audio file, and an optional prompt; the clip runs as long as the audio, and the prompt steers scene, emotion, and camera rather than the spoken words.

Recommended Settings

• •image: a clear, front-facing or three-quarter portrait with a closed or neutral mouth; higher resolution gives sharper output. Keep the aspect ratio between 0.4 and 2.5.
• •audio: clean speech or a vocal-forward song, mp3 or wav under 15 MB. Use 4–32s on /v1 (up to 112s on /v2) and trim silence; the video length equals the audio length.
• •prompt: describe scene, mood, framing, and camera — not the words. Add a named camera move to activate it. Six prompt languages are supported.

Example prompts to try

Each example below ran end to end at the settings shown, and the input portrait's identity, clothing, and background carried through the whole clip.

• •Expressive talking performance. Prompt: "A Victorian naturalist in a lamplit study lined with brass microscopes and glass specimen jars, holding up a fossil fern and leaning in as his expression shifts from calm to delight; medium close-up, steady framing." The mouth tracked the speech with varied shapes, the expression shifted calm to excited, and a hand rose into frame to gesture. Tip: keep the framing steady and the face front-facing for the cleanest lip-sync.
• •Singing, song-driven. Prompt: "A soul singer at a vintage ribbon microphone in a brass-trimmed jazz club, eyes closing on the held notes, one hand rising with the phrase; medium close-up, steady framing." Driven by a gospel-soul track, she formed sung vowel shapes and closed her eyes on the held notes — a real singing performance, not just a moving mouth. Tip: feed a track with clear lead vocals; instrumental-only audio produces little mouth motion.
• •Prompt-driven camera move. Prompt: "A horned fortune-teller in a candlelit tent hung with beads and tarot cards, leaning forward as her expression sharpens; the camera orbits around her while she speaks." The frame drifted and changed vantage around her while she delivered the line and gestured. Tip: name the move — orbit, push-in, or pan — explicitly in the prompt.

Further performances to try: "a lecturer explaining a diagram, handheld camera", "a children's book character singing a lullaby", "a news desk read with a slow push-in".

Does OmniHuman 2.0 clone or generate a voice?

No — you supply the audio, and the model animates to it.

Can I drive it with a video?

No. It is image-plus-audio only, not video-driven performance capture.

Can it sing?

Yes; it reads musical phrasing from the song and syncs sung notes.

What languages does the prompt support?

English, Chinese, Japanese, Korean, Spanish, and Indonesian.

How long can a clip be?

Up to 112 seconds on /v2; the video always matches your audio length.

What makes a good input image?

A sharp, front-facing or three-quarter portrait with an unobstructed, closed mouth and even lighting.