Luma Modify Video

Transform videos seamlessly with high-fidelity generative edits while preserving original actor performances.

API

If you're looking for an API, you can choose from your desired programming language.

POST

import requests

api_key = "YOUR_API_KEY"
url = "https://api.segmind.com/v1/modify-video"

# Prepare data and files
data = {}
files = {}

data['mode'] = "adhere_1"
data['prompt'] = "woman in a yarn doll style"
# For parameter "video_url", you can send a raw file or a URI:
# files['video_url'] = open('IMAGE_PATH', 'rb')  # To send a file
data['video_url'] = 'https://segmind-resources.s3.amazonaws.com/input/c08771b9-b671-4c12-9ea7-af4048b9d194-894a8bdf-6064-40ea-a78d-06c1abff262b.mp4'  # To send a URI
# For parameter "first_frame_url", you can send a raw file or a URI:
# files['first_frame_url'] = open('IMAGE_PATH', 'rb')  # To send a file
data['first_frame_url'] = 'https://segmind-resources.s3.amazonaws.com/input/f2220449-e53a-40d5-aba7-e12c6f562ab5-modify-video-ip.png'  # To send a URI

headers = {'x-api-key': api_key}

# If no files, send as JSON
if files:
    response = requests.post(url, data=data, files=files, headers=headers)
else:
    response = requests.post(url, json=data, headers=headers)
print(response.content)  # The response is the generated image

RESPONSE

image/jpeg

HTTP Response Codes

200 - OKImage Generated

401 - UnauthorizedUser authentication failed

404 - Not FoundThe requested URL does not exist

405 - Method Not AllowedThe requested HTTP method is not allowed

406 - Not AcceptableNot enough credits

500 - Server ErrorServer had some issue with processing

Attributes

modeenum:str ( default: adhere_1 )

How closely the output should follow the source video. Adhere: very close, for subtle enhancements. Flex: allows more stylistic change while keeping recognizable elements. Reimagine: loosely follows the source, for dramatic or transformative changes.

Allowed values:

promptstr ( default: 1 )

Guides video modification.

video_urlstr *

The source video URL. Use short MP4s under 30 seconds. Maximum video size is 100mb.

first_frame_urlstr ( default: 1 )

An optional URL of the first frame of the video. This should be a modified version of the original first frame, it will be used to guide the video modification.

To keep track of your credit usage, you can inspect the response headers of each API call. The x-remaining-credits property will indicate the number of remaining credits in your account. Ensure you monitor this value to avoid any disruptions in your API usage.

Modify Video – Generative Video Editing Model

What is Modify Video?

Modify Video is Luma AI’s state-of-the-art generative video editing model designed for high-fidelity transformations without reshooting. By leveraging advanced pose, facial and lip-sync tracking, it maintains full-body motion consistency while enabling scene swaps, environment retexturing and style restyling. Creators and developers can generate multiple variants from a single clip, each preserving natural temporal flow and actor performance signals.

Key Features

Full-Body & Facial Tracking: Retains original actor movements, expressions and lip-sync for seamless edits.
Temporal Consistency: Keeps frame-to-frame continuity, minimizing flicker and jitter in output.
Multiple Modes:
• adhere_1/adhere_2/adhere_3 – faithful, subtle edits
• flex_1/flex_2/flex_3 – balanced transformations
• reimagine_1/reimagine_2/reimagine_3 – creative overhauls
Prompt-Driven Styles: Use natural-language instructions (e.g., “make it vintage”) to guide restyling.
First-Frame Guidance: Supply a custom first frame image URL to anchor the visual style.
Native Resolution Support: Outputs at the same resolution as the source clip (MP4).

Best Use Cases

Scene Replacement: Swap backgrounds or entire sets in short clips under 30 seconds.
Creative Restyling: Transform modern footage into vintage, cinematic or genre-specific looks.
Lip-Sync Corrections: Fix dialogue sync issues while preserving actor performance.
Color Grading & Lighting Adjustments: Apply consistent color palettes and dynamic lighting effects.
Director-Grade Edits: Iterate on multiple stylistic variants quickly for pre-visualization and dailies.

Prompt Tips and Output Quality

Start Simple: Begin with clear prompts like “add neon lights” or “cinematic teal-orange grade.”
Adjust Mode:
- For minimal adjustments, choose adhere_1.
- For bold, artistic changes, use reimagine_1.
Leverage First Frame: Supply a stylized image via first_frame_url to guide overall look.
Clip Length: Keep source videos under 30 seconds for optimal processing speed.
Iterate Variants: Run the same prompt with different modes to compare subtle vs. dramatic effects.

FAQs

Q: What video formats are supported?
MP4 clips (under 30 seconds) are recommended for reliable ingestion and native resolution outputs.

Q: How do I choose between adhere, flex, and reimagine modes?
Use adhere modes for faithful edits, flex for moderate stylizations, and reimagine for full creative overhauls.

Q: Can I preserve lip-sync accuracy?
Yes—Modify Video’s facial tracking maintains lip-sync motion even when retexturing environments.

Q: Is a custom first frame required?
No. It’s optional but highly effective for guiding strong stylistic changes.

Q: How do I guide specific edits?
Set the prompt parameter with natural-language instructions (e.g., “make it look like film noir”) to direct the AI.

Other Popular Models

sdxl-img2img

SDXL Img2Img is used for text-guided image-to-image translation. This model uses the weights from Stable Diffusion to generate new images from an input image using StableDiffusionImg2ImgPipeline from diffusers

faceswap-v2

Take a picture/gif and replace the face in it with a face of your choice. You only need one image of the desired face. No dataset, no training

sdxl1.0-txt2img

The SDXL model is the official upgrade to the v1.5 model. The model is released as open-source software

sd1.5-majicmix

The most versatile photorealistic model that blends various models to achieve the amazing realistic images.