Every time you make a video call and the other person does not hear their own voice echoed back at them, a 66-year-old equation is doing that work. The Least Mean Squares algorithm, published by Bernard Widrow and his PhD student Marcian "Ted" Hoff at Stanford in 1960, is the mathematical core of acoustic echo cancellation in WebRTC, Zoom, Teams, FaceTime, and every hands-free phone call you have ever made.
That same Ted Hoff later went to Intel, where he designed the 4004, the world's first single-chip microprocessor. Before that, he helped invent one of the most deployed signal processing algorithms in history.
The Problem: A Speaker and a Microphone in the Same Room
Acoustic echo happens because physics. When a remote participant's voice comes through your speaker, that sound does not stop at your ears. It reflects off walls, bounces off your desk, and enters your microphone. The far-end participant hears their own voice arriving back with a delay of 20 to 200 milliseconds, the time it takes for sound to travel from speaker to microphone through the room.
Below roughly 20 ms, the brain perceives this as coloration rather than a distinct repeat. Above 50 ms, it becomes clearly perceptible as echo. Above 200 ms, it is as disruptive as a satellite phone delay. A typical office or home setup produces echo tails of 40 to 100 ms.
The naive fix is half-duplex: mute the microphone whenever the speaker is active. This works but feels unnatural, and it is exactly why speakerphones from the 1990s sound the way they do. The real solution is to model the echo path through the room, estimate what the echo looks like, and subtract it from the microphone signal before transmitting.
The Echo Path as an Unknown Filter
The AEC system has access to the signal being sent to the speaker, the far-end audio. The goal is to estimate what that signal looks like after traveling through the room and entering the microphone. That transformation is the acoustic echo path, and it behaves like a linear time-varying filter.
At any moment, the echo path can be represented as a finite impulse response (FIR) filter: a list of coefficients, one per sample, that describes how the room transforms the speaker output into the echo at the microphone. At a 16 kHz sample rate, a 640-tap FIR filter models 40 ms of echo tail. Larger rooms with significant reverberation require longer filters, sometimes 2048 taps or more to cover 128 ms.
The problem is that these coefficients are unknown and change constantly. Someone moves a laptop, opens a window, or shifts in their chair, and the echo path shifts. The filter must learn it in real time, from streaming audio, without ever seeing the correct answer.
The LMS Algorithm
In 1960, Widrow and Hoff published "Adaptive Switching Circuits" at the IRE WESCON Convention. Their paper introduced the ADALINE (Adaptive Linear Neuron) network, trained by a new learning rule: the Least Mean Squares algorithm. LMS adjusts filter coefficients iteratively to reduce the difference between predicted and observed output. The update rule is:
w[n+1] = w[n] + 2μ · e[n] · x[n]w[n] is the coefficient vector. x[n] is the reference input, the speaker signal. e[n] is the error: the difference between the actual microphone signal and the filter's estimated echo output. μ (mu) is the step size, a small positive constant controlling the learning rate.
Each sample, the algorithm nudges each coefficient slightly in the direction that reduces the current error. Over hundreds of samples, the coefficient vector converges toward the actual echo path impulse response. Once converged, the estimated echo is subtracted before transmission and the far-end speaker hears silence rather than their own voice.
The tradeoff in μ is fundamental: a large step size converges fast but oscillates around the true solution, leaving residual echo. A small step size converges slowly but reaches a more stable estimate. Neither extreme is universally correct.
Normalized LMS: Making Step Size Adaptive
Standard LMS misbehaves when signal levels vary. A loud transient drives adaptation too aggressively, potentially diverging. A quiet passage stalls convergence. NLMS (Normalized LMS) solves this by dividing the step size by the instantaneous power of the input:
w[n+1] = w[n] + (μ / ||x[n]||²) · e[n] · x[n]Now the effective learning rate automatically decreases when the signal is loud and increases when it is quiet. This makes NLMS substantially more stable across the 40 dB or more of dynamic range that normal speech occupies. Most production AEC implementations use NLMS or a variant with variable step size that adjusts based on estimated convergence state.
The Double-Talk Problem
Everything above assumes only the far-end speaker is active. When both parties speak simultaneously, what engineers call double-talk, the adaptive filter sees a microphone signal that contains both echo and near-end speech. LMS cannot tell them apart. It treats near-end speech as an error signal to be minimized, and starts modifying coefficients to cancel it. The filter diverges from the correct echo path and the system falls apart.
The solution is a double-talk detector (DTD): a separate algorithm that monitors the microphone for evidence of near-end speech activity. When near-end speech is detected, adaptation freezes. The coefficient vector holds its last-known-good values while both parties speak, then resumes adapting when the near-end speaker stops.
Detecting double-talk reliably is harder than it sounds. A cross-correlation detector compares the similarity between the microphone signal and the filtered reference; near-end speech reduces this correlation. Frequency-domain coherence measures are more sensitive. Modern implementations often use several detectors in parallel, with a state machine managing transitions between adapt, hold, and residual-suppression modes.
After LMS: The Post-Filter
LMS convergence is not perfect. Even a well-converged filter leaves residual echo, typically 20 to 40 dB below the original level but still audible in quiet passages. A post-filter handles nonlinear residual suppression: a spectral estimator measures the remaining echo power in each frequency band and applies gain reduction where echo energy persists. This is closer to noise reduction than echo cancellation, and it is why processed audio often sounds slightly attenuated during pauses in near-end speech.
WebRTC AEC3, the open-source echo cancellation module used in Chrome, Firefox, and applications built on the WebRTC stack, uses exactly this architecture: an NLMS adaptive filter feeding into a frequency-domain post-filter, with a comfort noise generator to fill silence without an unnaturally dead noise floor.
AEC Is Not ANC
These abbreviations are often confused. Active noise cancellation (ANC), the technology in headphones like AirPods Pro, is a different problem. ANC removes external ambient noise using a microphone on the outside of the earcup. It hears incoming noise, inverts the phase, and plays the inverted signal into the ear to cancel the arriving sound. The acoustic path is short and relatively stable: from the outer mic to the driver inside the cup, a few centimeters.
AEC, by contrast, models an unknown and changing transmission path through an arbitrary room, trains an adaptive filter against a reference signal, and subtracts a learned estimate of the echo. The scale difference is fundamental: ANC operates over centimeters and milliseconds, AEC over meters and tens of milliseconds.
Why This Still Runs in Production
The LMS algorithm is computationally inexpensive. At 16 kHz with 640 taps, NLMS requires roughly 640 multiply-accumulate operations per sample, around 10 million operations per second. Modern processors handle this without measurable load. The challenge is not compute; it is robustness.
Echo paths change faster than simple NLMS converges. Rooms contain nonlinear elements such as speaker saturation at high volumes, which pure LMS cannot model. Music-content signals have long-range correlations that confuse the filter's sense of convergence. Smart speakers must handle far-field conditions where the echo path is 3 to 5 meters long and highly reverberant.
Modern AEC research is less about replacing LMS and more about surrounding it: machine learning classifiers that recognize when adaptation has gone wrong, neural post-filters trained on large datasets of speech and echo mixtures, and multi-channel beamforming that reduces the echo before the adaptive filter even starts.
Sixty-six years after Widrow and Hoff's IRE WESCON paper, the core update equation is unchanged. What surrounds it has grown considerably more complex.