Stem Separation Explained: How AI Isolates Vocals and Instruments

Stem separation — also called source separation — is the process of decomposing a mixed audio track into individual instrument stems. If you have a finished stereo mix and want just the vocal line, or the drum pattern, or the bass groove, stem separation is how you get there without the original multitrack session.

How Demucs Works

Demucs is an open-source deep neural network model developed by Meta Research that uses a U-Net architecture operating on both the raw waveform and the spectrogram simultaneously. Unlike earlier FFT-based approaches that created audible artifacts (the classic 'phasey' sound), Demucs processes temporal dependencies in the audio signal, which dramatically reduces the musical noise floor in separated stems.

• Waveform encoder: compresses the raw audio into a learned latent representation.
• Spectrogram encoder: simultaneously processes the frequency-domain view of the same signal.
• Dual-path transformer: models long-range dependencies across both representations.
• Decoder: reconstructs each stem from the shared latent space.

The Four Standard Stems

Demucs v4 (htdemucs) separates a stereo mix into four stems by default: vocals, drums, bass, and other (everything else — guitars, keys, synths, orchestral elements). Each stem is output as a separate stereo file at the original sample rate.

Vocals

The vocals stem isolates lead and backing vocals. Quality degrades when the vocal sits very close in frequency to a sustained synthesizer pad — the model cannot always distinguish sustained harmonic content from voice formants. For most commercial pop, R&B, and hip-hop material, vocal isolation quality is production-usable.

Drums

Drums are the most reliably separated stem because percussion has a distinctive transient profile that is easy for the model to identify. Kick, snare, hi-hats, and cymbals all separate well unless the mix has heavy reverb smearing the transients.

Practical Uses in Music Production

• Isolate the vocal from a reference track to study the performance style.
• Extract the vocal stem to make an acapella for a DJ edit.
• Remove the bass from a full mix to re-record it with a different instrument.
• Create an instrumental version of a track where the original multitrack no longer exists.
• Transcribe a melody by separating the lead instrument and running it through pitch detection.

Running Demucs Locally Without Uploading Anything

Cloud-based stem separation tools such as LALAL.AI and Moises work on files you upload to them: LALAL.AI's privacy policy says its algorithms "analyze, process, and separate existing user-uploaded audio files", and Moises' says it keeps "your media and processed files" (both read on 2026-10-22). For unreleased material — demos, client work, sync licensing tracks — read their retention terms before you upload. edytlab runs Demucs as a local tool call: the model runs on your machine, the stems are written into your project folder, and nothing is uploaded. The model (about 171 MB) is a one-time download you start from Settings, and the tool needs about 4 GB of free memory.

The workflow in edytlab is one sentence: "separate the stems on track 1". The agent calls the stem separation tool and Demucs runs on-device. The four stems are written as files in your project, and the session is not changed until you ask the agent to load the one you want, which becomes a track in your timeline.

Model Selection and Quality Trade-offs

Demucs offers several model variants. htdemucs is the recommended default — it offers the best quality-to-speed ratio on modern hardware. mdx_extra gives slightly better vocal quality at the cost of more VRAM. htdemucs_6s adds guitar and piano as separate stems, which is useful for complex arrangements but takes longer to run. That is Demucs in general: edytlab's separate_stems recognises only htdemucs, and rejects any other model id.

How long separation takes depends on your hardware and on the variant, and a GPU helps a great deal. edytlab runs on the CPU: on a four-core computer a track takes about half its own length. The stems are estimates, so each one carries a little of the others.

edytlab is an open-source, local-first AI audio editor. Download the latest release or star it on GitHub.