Sam Audio Large

SAM-Audio: Audio Source Separation Model

What is SAM-Audio?

SAM-Audio is a foundation AI model from Meta designed for audio source separation: isolating a target sound (like “drums”, “speech”, or “siren”) from a mixed recording. Instead of separating only fixed stems, SAM-Audio aims to segment any sound you describe, making it useful for modern audio editing pipelines, sound event detection, and multimedia analysis.

On Segmind, you provide an audio input and a sound description prompt. The model returns an isolated track containing the requested source, enabling workflows like “extract vocals from a song”, “remove background noise”, or “pull out footsteps from a scene”.

Key Features

• •Prompted sound isolation using natural language text (e.g., “keyboard typing”, “female narration”).
• •Fine-grained separation for complex mixes (music, ambience, dialogue + effects).
• •Developer-friendly inputs: audio via URL or Base64, with selectable output format.
• •Quality tuning with reranking to improve the best-candidate separation.
• •Strong fit for automated pipelines (moderation, indexing, annotation, post-production).

Best Use Cases

• •Audio editing & post-production: isolate dialogue, ambience, SFX, instruments.
• •Content creation: remixing, stem-like extraction, cleaner voiceovers.
• •Sound event detection: extract target events before classification or labeling.
• •Multimedia & video analysis: separate scene sounds for search and retrieval.
• •Accessibility: enhance speech tracks for transcription and captioning.

Prompt Tips and Output Quality

• •Be specific: “snare drum hits” often separates better than “drums”.
• •Include context: “crowd cheering in a stadium” vs. “cheering”.
• •If multiple similar sources exist, add qualifiers: “lead vocal”, “background chatter”.
• •Use output_format: wav for highest fidelity; mp3 for smaller files.
• •Increase reranking_candidates (1–8) when the separation is close but imperfect; higher values typically improve selection at the cost of more computation.

Core parameters

• •audio (required): URL/Base64 for the input audio.
• •description (required): the sound to isolate.
• •output_format (optional): wav or mp3 (default wav).
• •reranking_candidates (optional, advanced): candidate count (default 4).

SAM-Audio: Audio Source Separation Model

Unlock prompt-driven sound isolation for music production, post-production, content creation, and analysis.

Quick Start

1. •Prepare Input
   • audio: URL or Base64 of your mixed track (e.g., https://.../office.mp4).
   • description: concise noun phrase (e.g., “snare hits”, “female narration”).
2. •Choose Output
   • output_format: wav (highest fidelity) or mp3 (smaller footprint).
3. •Adjust Quality
   • reranking_candidates (1–8, default 4): more candidates increase separation quality at cost of compute.

Core Parameters

{
  "audio": "...",              
  "description": "drums",      
  "output_format": "wav",      
  "reranking_candidates": 4    
}

Recommended Settings by Use Case

• •

  Podcast & Voiceovers
  • description: “male speech in studio”
  • output_format: wav
  • reranking_candidates: 3 (fast) or 5 (cleaner)

• •

  Music Remix & Stems
  • description: “lead vocals”, “hi-hat groove”
  • output_format: wav
  • reranking_candidates: 6–8

• •

  Sound Effects for Video
  • description: “footsteps on gravel”, “thunderstorm ambience”
  • output_format: mp3 (preview) or wav (final)
  • reranking_candidates: 4

• •

  Noise Reduction & Forensics
  • description: “car engine noise”, “airplane cabin hum”
  • output_format: wav
  • reranking_candidates: 7–8

• •

  Event Detection Preprocessing
  • description: “dog barking”, “gunshot”
  • output_format: mp3
  • reranking_candidates: 2–4

Tips & Best Practices

• •Be specific: “snare drum hits” vs. “drums”.
• •Add context: “crowd cheering in stadium” improves focus.
• •Qualify overlaps: “background chatter” vs. “lead dialogue”.
• •Preview with mp3, finalize in wav.
• •Increase reranking_candidates only if output artifacts persist.

Troubleshooting

• •Artifact speckles: lower candidates or revise description.
• •Missing detail: raise candidates to 6–8.
• •Slow runtime: use candidates 2–4.

Leverage SAM-Audio’s promptable interface for bespoke audio pipelines— from automated indexing to professional post-production.

Is SAM-Audio open-source?

Meta publishes research assets for SAM-Audio, but licensing and usage terms may vary by distribution. Check the upstream repository/terms for your deployment scenario.

How is SAM-Audio different from stem splitters (vocals/drums/bass)?

It’s prompt-driven: you can target any described sound, not only fixed music stems.

What should I put in description for best results?

Use a concise noun phrase plus qualifiers (instrument, source, environment), e.g., “male speech in a car”, “dog barking”, “hi-hat pattern”.

Should I choose WAV or MP3 output?

Choose WAV for editing and evaluation; choose MP3 for lightweight previews and distribution.

What does reranking_candidates do?

It controls how many separation candidates are generated and reranked; increasing it can improve the final isolated track when prompts are ambiguous.