Live Portrait video to video

Experience the magic of Live Portrait’s Video-to-Video Model! Transform your static images into dynamic videos seamlessly.

Playground

Try the model in real time below.

live_portrait_v2v_input_vid.mp4 selected

You can drop your own file here

livie_portrait_driving_vid.mp4 selected

You can drop your own file here

Set to 'true' to return the audio of the driving video; if 'false', the audio of input video will be returned.



Examples

Check out what others have created with Live Portrait video to video

API

If you're looking for an API, you can choose from your desired programming language.

POST
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 import requests import base64 # Use this function to convert an image file from the filesystem to base64 def image_file_to_base64(image_path): with open(image_path, 'rb') as f: image_data = f.read() return base64.b64encode(image_data).decode('utf-8') # Use this function to fetch an image from a URL and convert it to base64 def image_url_to_base64(image_url): response = requests.get(image_url) image_data = response.content return base64.b64encode(image_data).decode('utf-8') api_key = "YOUR_API_KEY" url = "https://api.segmind.com/v1/live-portrait-video-to-video" # Request payload data = { "input_video": "https://segmind-sd-models.s3.amazonaws.com/display_images/live-portrait-v2v/live_portrait_v2v_input_vid.mp4", "driving_video": "https://segmind-sd-models.s3.amazonaws.com/display_images/live-portrait-v2v/livie_portrait_driving_vid.mp4", "dsize": 512, "scale": 2.3, "driving_audio": False, "vx_ratio": 0, "vy_ratio": -0.125, "input_face_index": 0, "drive_face_index": 0, "crop_drive_face": False, "lip_zero": True, "lip_zero_threshold": 0.03, "eye_retargeting": False, "eyes_retargeting_multiplier": 1, "lip_retargeting": False, "lip_retargeting_multiplier": 1, "stitching": True, "relative": True, "mismatch_method": "cut", "video_frame_load_cap": 120, "base64": False } headers = {'x-api-key': api_key} response = requests.post(url, json=data, headers=headers) print(response.content) # The response is the generated image
RESPONSE
image/jpeg
HTTP Response Codes
200 - OKImage Generated
401 - UnauthorizedUser authentication failed
404 - Not FoundThe requested URL does not exist
405 - Method Not AllowedThe requested HTTP method is not allowed
406 - Not AcceptableNot enough credits
500 - Server ErrorServer had some issue with processing

Attributes


input_videostr *

Input video


driving_videostr *

Driving video


dsizeint *

Size of the video

min : 64,

max : 2048


scalefloat *

Scale of the video

min : 1,

max : 4


driving_audiobool ( default: 1 )

Set to 'true' to return the audio of the driving video; if 'false', the audio of input video will be returned.


vx_ratiofloat ( default: 1 )

Horizontal ratio for transformation

min : -1,

max : 1


vy_ratiofloat ( default: -0.125 )

Vertical ratio for transformation

min : -1,

max : 1


input_face_indexint *

Index of the input face

min : 0,

max : 5


drive_face_indexint *

Index of the driving face

min : 0,

max : 5


crop_drive_facebool ( default: 1 )

Crop the driving face


lip_zerobool ( default: true )

Zero out the lips


lip_zero_thresholdfloat ( default: 0.03 )

Threshold for zeroing out the lips

min : 0,

max : 5


eye_retargetingbool ( default: 1 )

Enable eye retargeting


eyes_retargeting_multiplierfloat ( default: 1 )

Multiplier for eye retargeting

min : 0.01,

max : 10


lip_retargetingbool ( default: 1 )

Enable lip retargeting


lip_retargeting_multiplierfloat ( default: 1 )

Multiplier for lip retargeting

min : 0.01,

max : 10


stitchingbool ( default: true )

Enable stitching


relativebool ( default: true )

Use relative method


mismatch_methodenum:str *

Method for mismatch handling

Allowed values:


video_frame_load_capint *

The maximum number of frames to load from the driving video. Set to 0 to use all frames.


base64boolean ( default: 1 )

Base64 encoding of the output image.

To keep track of your credit usage, you can inspect the response headers of each API call. The x-remaining-credits property will indicate the number of remaining credits in your account. Ensure you monitor this value to avoid any disruptions in your API usage.


Pricing

Serverless Pricing

Buy credits that can be used anywhere on Segmind

$ 0.0038 /per second

Dedicated Cloud Pricing

For enterprise costs and dedicated endpoints

$ 0.0007 - $ 0.0031 /per second
FEATURES

PixelFlow allows you to use all these features

Unlock the full potential of generative AI with Segmind. Create stunning visuals and innovative designs with total creative control. Take advantage of powerful development tools to automate processes and models, elevating your creative workflow.

Segmented Creation Workflow

Gain greater control by dividing the creative process into distinct steps, refining each phase.

Customized Output

Customize at various stages, from initial generation to final adjustments, ensuring tailored creative outputs.

Layering Different Models

Integrate and utilize multiple models simultaneously, producing complex and polished creative results.

Workflow APIs

Deploy Pixelflows as APIs quickly, without server setup, ensuring scalability and efficiency.

Live Portrait Video-to-Video

The Live Portrait Video Model is a robust deep learning tool designed to facilitate the generation of realistic video portraits. This model leverages advanced neural network architectures to convert an input video of a subject and a driving video into a seamlessly animated output. It captures subtle facial expressions and movements, ensuring that the resulting video maintains a high level of accuracy and realism.

How to Use Live Portrait Video?

  1. Input Video: Upload the source video featuring the subject whose portrait you want to animate.

  2. Driving Video: Upload the driving video that contains the desired expressions and movements.

  3. Generate Output: Click on the "Generate" button to create the animated portrait video.

How to Fine-tune video outputs with Live Portrait Video?

The model offers several parameters that can be fine-tuned to achieve desired outputs:

Input Face Index

Utilize the Input Face Index when the input video contains multiple faces, and you need to specify which face to animate. Identify the index of the face in the input video. Set the Input Face Index parameter to the corresponding index number.

Drive Face Index

Similar to the Input Face Index, use the Drive Face Index when the driving video contains multiple faces. Determine the index of the desired face in the driving video. Set the Drive Face Index parameter to the appropriate index number

Mismatch Method

Use the Mismatch Method when there are significant differences between the facial features of the input and driving videos. Choose from available options like 'cut', 'blend', etc. Select the method that best handles the discrepancies ensuring smooth animation.

Video Frame Load Gap

Use this option to manage the trade-off between processing speed and video smoothness. Set a lower value for smoother animation. Set a higher value to speed up processing if small frame gaps are acceptable.

Crop Drive Face

Use Crop Drive Face when you need to focus on a particular area of the driving face, removing unnecessary elements. The model will automatically crop the driving face.

Lip Zero

Use the Lip Zero parameter to control the sensitivity of lip movements in the generated video. Adjust the Lip Zero Threshold to the desired sensitivity level. Lower values make small lip movements more prominent.

Eye Retargeting

Enable eye retargeting to achieve more realistic eye movements in the animated portrait. Set the Eyes Retargeting Multiplier to control the intensity of eye movements.

Lip Retargeting

Enable lip retargeting to replicate the driving video’s lip movements accurately in the animated portrait. Set the Lip Retargeting Multiplier to adjust the extent of lip movement replication.

Stitching

Use stitching to smooth transitions between frames, particularly when there are noticeable seams or discontinuities. The model will apply stitching techniques to create seamless transitions.

Relative Method

Utilize the Relative Method for finer control over the animation, typically when exact alignment with the driving video is not required. The model will use relative positioning to make nuanced adjustments, allowing for more fluid animation.

For optimal results, adjust these parameters based on the specific requirements of your project and the characteristics of your input and driving videos.

Use-cases

The Live Portrait Video Model is ideal for applications in the fields of animation, entertainment, and communication. It can be used for creating animated portraits for films, games, virtual influencers, educational content, and more. The model ensures high-quality and realistic animations that can enhance user engagement in various multimedia projects.

F.A.Q.

Frequently Asked Questions

Take creative control today and thrive.

Start building with a free account or consult an expert for your Pro or Enterprise needs. Segmind's tools empower you to transform your creative visions into reality.

Pixelflow Banner