Most people who use AI stem separation eventually notice the same thing. The vocal comes out usable, but it carries a faint warble, a smeared, slightly underwater quality that was not in the original. The instinct is to blame the model size or the training data. The real cause is older and more stubborn. It is a single decision about how the math handles phase, and it puts a ceiling on quality that no amount of training fully removes.
What an iSTFT actually needs
Spectrogram-based separators do their work in the frequency domain. They take the mixture, run a Short-Time Fourier Transform (STFT), process the result, then run an inverse STFT (iSTFT) to get audio back.
The STFT of a signal is complex. Every time-frequency bin holds two numbers, not one: a magnitude (how much energy sits at that frequency in that moment) and a phase (where in its cycle that frequency component is). The magnitude is the part you see when you look at a spectrogram. The phase is invisible in that picture, but it is half the information.
The iSTFT needs both numbers to reconstruct audio. Drop the phase and you cannot get a waveform back. This is the constraint the whole artifact problem hangs on.
The shortcut
Here is what a classic spectrogram separator like Spleeter actually predicts. Spleeter, released open source by Deezer on 4 November 2019, is a twelve-layer U-Net, six encoder layers and six decoder layers, operating on the magnitude spectrogram. For each source it outputs a soft mask: a grid the same size as the spectrogram where every cell holds a value between 0 and 1, meaning "how much of the energy in this bin belongs to the vocal." Multiply the mask by the mixture magnitude and you have the estimated magnitude of the isolated source.
Notice what is missing. The network predicted a magnitude. It never predicted a phase. So to run the iSTFT and get audio out, the separator borrows the phase of the original mixture and applies it to the new magnitudes.
That borrowing is the shortcut. It is cheap, it is fast, and it is lossy.
Why the borrowed phase is wrong
Consider a single time-frequency bin where a vocal note and a guitar note happen to overlap. In the original mix, that bin has exactly one phase value, the result of the two sources adding together. When you split the bin into a vocal estimate and a guitar estimate, each isolated source should have its own phase. But you only have the one mixed phase to hand out, so you give the same value to both.
It cannot be correct for either of them. The phase information that would tell the two sources apart was never stored separately in the first place. It is physically gone the moment the two signals sum in the recording.
This is not a soft limitation. The academic literature is direct about it: applying the mixture phase to estimated source magnitudes "imposes an upper bound on the performance of the system," and the recovered sources "inherently exhibit audible phase artifacts." Because the masks are bounded in the range 0 to 1 and reuse the mixture phase, the best achievable separation quality is capped before training even begins. The warble, the comb-filter coloration, the underwater sheen all trace back to this one approximation.
It also explains which material separates worst. The more two sources overlap in the same bins, the more shared phase you have to fake. Dense arrangements, doubled or harmonized vocals, and synth pads that smear energy across the spectrum are exactly the cases where the shortcut hurts most. Sparse, well-separated material hides the problem because there is less overlap to misattribute.
Trying to claw the phase back
The obvious response is to estimate a better phase instead of copying the mixture. The classic tool is the Griffin-Lim algorithm, which bounces a signal between STFT and iSTFT repeatedly, nudging the phase each pass so it becomes consistent with the target magnitude. For source separation there is a variant called MISI (Multiple Input Spectrogram Inversion) that adds a useful constraint: all the estimated sources, with their estimated phases, must add back up to the original mixture.
Both work, up to a point. Both are also iterative, which makes them slow, and neither fully recovers information that was never there. Phase reconstruction is a plausible guess, not a retrieval. You can make the warble quieter. You cannot make it disappear by guessing harder.
The cleaner fix: never throw phase away
The other way out is to stop using a spectrogram at all. This is the path Demucs took. From Meta AI (Alexandre Defossez and colleagues, MIT licensed), Demucs operates directly on the raw waveform with a U-Net style encoder and decoder. Because it never converts to a magnitude spectrogram, it never discards phase, so there is nothing to borrow back later.
The current state of the art is hybrid rather than purely one or the other. Hybrid Transformer Demucs (HT Demucs, arXiv 2211.08553) runs two parallel branches, one on the waveform and one on the spectrogram, and uses a Transformer to share information between them. It reaches roughly 9.0 dB SDR on MUSDB-HQ without extra training data and about 9.2 dB with additional data and per-source fine-tuning. The earlier Hybrid Demucs won Sony's 2021 Music Demixing Challenge. Worth noting: the Demucs repository is no longer actively maintained, even though the models remain widely used through front ends like Ultimate Vocal Remover.
SDR here is Signal-to-Distortion Ratio in decibels, higher being cleaner, measured on MUSDB18, a benchmark of 150 songs split 100 for training and 50 for testing, each with isolated bass, drums, vocals and an "other" bucket. The number to take away is the ceiling. Even the best vocal separation tops out around 9 dB SDR. That is clean enough for remixing, sampling and practice tracks. It is not clean enough to pass as the untouched original, and the phase story is a large part of why.
What this means in practice
A few things follow directly from the mechanism.
Match the model to the source. Time-domain and hybrid models such as Demucs preserve transients better, so drums come out crisper. Older spectrogram-only tools smear percussion because the phase smearing hits sharp transients hardest.
Treat reverb as the enemy. A dry, well-recorded vocal separates far better than a wet one. The reverb tail occupies bins shared with other sources, so it either bleeds into adjacent stems or gets split off with the vocal. Expect to add reverb back after separation rather than relying on the model to keep it intact.
Do not fight dense or synth-heavy material. Broadband pads and harmonized vocals occupy too much shared spectrum, which is precisely the shared-phase case. Use separation for jobs where small imperfections hide under new production, like sampling and remixing, not for forensic isolation where every flaw is exposed.
None of this is a knock on the technology. It is a reminder that the artifacts are structural, not a bug waiting for a patch. The field did not get better at stem separation by predicting phase more cleverly. It got better by building models that never threw the phase away to begin with.