BSS-Eval: Metrics for Blind Source Separation

• BSS-Eval is a framework that decomposes an estimated source into target, interference, and artifact components by projecting onto reference subspaces.
• It computes key metrics such as SDR, SIR, and SAR—with extensions like ISR and SI variants—to quantify overall distortion, interference, and artifacts.
• Recent advances integrate fast computation, referenceless prediction, and perceptual metrics to address limitations in evaluating nonlinear or generative source separation methods.

Blind Source Separation Evaluation (BSS-Eval) denotes the standard family of objective metrics used to judge blind source separation systems. In its canonical reference-based form, it evaluates an estimated source by decomposing it into components attributable to the target reference, to other reference sources, and to a residual artifact term, yielding scores such as signal-to-distortion ratio (SDR), signal-to-interference ratio (SIR), and signal-to-artifact ratio (SAR). Adjacent literature also reports source image-to-spatial distortion ratio (ISR) and scale-invariant variants such as SI-SDR, SI-SIR, and SI-SAR. Recent work retains BSS-Eval as the established baseline, while increasingly treating it as a limited proxy for perceptual quality in musical and generative source separation (Scheibler, 2021, Bereuter et al., 22 Apr 2026, Bereuter et al., 15 Jul 2025).

1. Canonical reference-based decomposition

The classic BSS-Eval procedure models an estimated source $\hat{\vs}_m$ as a sum of filtered references plus residual error,

$\hat{\vs}_m = \sum_k \vh_{mk} \star \vs_k + \vb_m,$

where $\vh_{mk}$ is a distortion filter relating reference $\vs_k$ to estimate $\hat{\vs}_m$, $\star$ denotes convolution truncated to the signal length, and $\vb_m$ is the residual. The framework allows some mismatch between estimate and reference through a distortion filter of length $L$, often $L=512$. This makes the evaluation more forgiving than strict samplewise comparison, since an estimate may still be rated favorably when it differs by linear filtering or reverberation-like effects (Scheibler, 2021).

For each reference–estimate pair $(k,m)$, the estimate is decomposed into target, interference, and artifact components: $\hat{\vs}_m = \sum_k \vh_{mk} \star \vs_k + \vb_m,$0 Here $\hat{\vs}_m = \sum_k \vh_{mk} \star \vs_k + \vb_m,$1 is the orthogonal projector onto the subspace spanned by the $\hat{\vs}_m = \sum_k \vh_{mk} \star \vs_k + \vb_m,$2 shifted copies of reference $\hat{\vs}_m = \sum_k \vh_{mk} \star \vs_k + \vb_m,$3, and $\hat{\vs}_m = \sum_k \vh_{mk} \star \vs_k + \vb_m,$4 is the orthogonal projector onto the subspace spanned by all shifted copies of all references. If $\hat{\vs}_m = \sum_k \vh_{mk} \star \vs_k + \vb_m,$5 contains the $\hat{\vs}_m = \sum_k \vh_{mk} \star \vs_k + \vb_m,$6 shifts of $\hat{\vs}_m = \sum_k \vh_{mk} \star \vs_k + \vb_m,$7 and $\hat{\vs}_m = \sum_k \vh_{mk} \star \vs_k + \vb_m,$8, the projectors are

$\hat{\vs}_m = \sum_k \vh_{mk} \star \vs_k + \vb_m,$9

The standard scores are then

$\vh_{mk}$0

$\vh_{mk}$1

$\vh_{mk}$2

After the pairwise scores are computed, the best permutation between estimates and references is typically found by the Hungarian algorithm (Scheibler, 2021).

This reference-subspace view underlies the later characterization of BSS-Eval as a projection-based metric family. In the singing voice separation literature, the same idea is summarized as decomposition of estimation error by projecting the estimated source onto FIR-filtered subspaces of the target and distorting sources (Bereuter et al., 15 Jul 2025).

2. Metric family, interpretation, and scale variants

BSS-Eval is used in the literature as a family rather than a single scalar. Different papers emphasize different members of the family, depending on whether the task foregrounds overall distortion, source leakage, artifacts, or spatial/filtering effects (Grais et al., 2018, Bereuter et al., 22 Apr 2026).

Metric	Interpretation in the cited literature	Notes
SDR	Overall distortion, combining impairments	Often treated as a global quality measure
SIR	Interference from other sources	Higher SIR means better suppression of source leakage
SAR	Artifact component of separation error	Larger SAR means fewer artifacts
ISR	Spatial/filtering distortion	Reported in BSS-Eval summaries, especially multichannel settings
SI-SDR / SI-SIR / SI-SAR	Scale-invariant variants	Included in MSS comparisons as variants from Roux et al.

Within this family, SDR is described as the global quality measure obtained by projecting the estimation error onto the target subspace, while SIR and SAR decompose that error into interference and artifact components. In another summary of BSS-Eval, the error between reference and estimate is described as decomposed into target distortion, artifacts, and interference, with ISR, SAR, SIR, and SDR as the corresponding energy ratios (Grais et al., 2018, Bereuter et al., 22 Apr 2026).

The role of ISR is more specialized in the provided literature. It is presented as source image-to-spatial distortion ratio, interpreted as a measure of spatial or filtering distortion, whereas SDR, SIR, and SAR remain the principal waveform-quality indicators in most single-channel and musical-source-separation evaluations (Grais et al., 2018, Bereuter et al., 15 Jul 2025).

Scale handling has become a practical extension point. One musical source separation study explicitly includes SI-SDR, SI-SIR, and SI-SAR alongside SDR, SIR, and SAR, treating the scale-invariant versions as established variants rather than as a conceptual replacement of the original framework (Bereuter et al., 22 Apr 2026).

3. Geometric interpretation and fast computation

A later reformulation shows that the BSS-Eval outputs are fully specified by just two projection energies, which are themselves squared cosines of two angles between estimate and reference subspaces. Defining

$\vh_{mk}$3

the metrics can be written as

$\vh_{mk}$4

with

$\vh_{mk}$5

Under the normalization $\vh_{mk}$6, $\vh_{mk}$7 and $\vh_{mk}$8 are squared cosines of angles between the estimate and the relevant target or full-reference subspaces. This makes explicit that SDR depends on projection onto the target-source subspace, SIR on the relation between target and full-reference projections, and SAR on how much of the estimate is explainable by any reference source rather than by the residual artifact term (Scheibler, 2021).

The same work argues that standard implementations are slowed primarily by large structured linear systems. The SDR-related system is Toeplitz, while the SIR-related system is block-Toeplitz. A fast implementation avoids explicit reconstruction of target, interference, and artifact waveforms and instead computes the scalar energies $\vh_{mk}$9 and $\vs_k$0. The main acceleration uses conjugate gradient descent with circulant preconditioning and FFT-based matrix–vector products. The complexity of the structured solve is thereby reduced by a factor quadratic in the distortion filter size used in BSS-Eval, usually 512, and the reported speed-up is up to two orders of magnitude in some cases (Scheibler, 2021).

Accuracy is reported to remain acceptable for most applications. The fp64 implementation agrees with a reference implementation to within less than $\vs_k$1 dB on CPU, fp32 typically remains within $\vs_k$2 to $\vs_k$3 dB, and a 10-iteration conjugate-gradient approximation gives median errors below $\vs_k$4 dB, with some outliers above $\vs_k$5 dB appearing more often for SAR. The same implementation is reported to support neural-network training, and longer distortion filters may be beneficial, with $\vs_k$6 outperforming $\vs_k$7 in the reported training study (Scheibler, 2021).

4. Reference dependence and referenceless approximation

A central limitation of BSS-Eval is that all of its standard scores depend on comparison with clean reference sources. Without reference sources, the toolkit cannot compute ISR, SAR, SIR, or SDR. This limits direct applicability in real-world situations where the ground truth audio is unavailable (Grais et al., 2018).

One response is referenceless prediction of BSS-Eval quantities. A 2018 study proposes a deep neural network that does not compute BSS-Eval directly, but learns to predict the BSS-Eval SAR score from the separated signal alone. During training, the true SAR is computed with references and used as a regression target; at inference time, the network maps features from the separated audio into an estimated SAR $\vs_k$8, with parameters trained by minimizing mean-square error between predicted SAR and BSS-Eval SAR (Grais et al., 2018).

The system uses mel-frequency spectrogram features from the separated source: stereo is converted to mono by averaging the channels, a spectrogram is computed, the spectrogram is converted to a mel-frequency spectrogram with 128 frequency channels, and 40 neighboring frames are stacked, yielding $\vs_k$9 input values. The regression model is a fully connected feed-forward DNN with 3 hidden layers, 500 nodes per hidden layer, ReLU activations in the hidden layers, and a linear activation in the output layer. Because musical quality changes over time, SAR targets are computed every 117 ms using a 464 ms window on 116 s excerpts, so the model predicts a time-varying SAR rather than a single global song-level score (Grais et al., 2018).

On the SiSEC-2016-MUS-task dataset, performance is reported in three scenarios of increasing difficulty: within-algorithm, across-known-algorithms, and across-unknown-algorithms. Mean absolute error and correlation against reference-based BSS-Eval SAR degrade as generalization becomes harder, from 1.2 dB and 0.84 correlation in the within-algorithm condition to 2.3 dB and 0.74 correlation for unknown algorithms. This establishes a practical distinction between traditional BSS-Eval, which is accurate but reference-dependent, and referenceless approximation, which is deployable but approximate (Grais et al., 2018).

5. Benchmark role in separation research

Beyond its role as an evaluation toolkit, BSS-Eval functions as an objective benchmark that shapes algorithm comparison, post-processing, and even source-image estimation.

In dual-channel, two-source convolutive BSS with the TRINICON framework, the BSS-Eval toolkit is used for objective evaluation through SIR and SDR. Higher SIR is interpreted as better suppression of interfering source leakage, and higher SDR as less overall distortion relative to the clean target speech. In that study, a two-pass TRINICON method with multi-source activity detection usually improves both metrics relative to normal frequency-domain TRINICON, with improvements largest when the two sources are in the same half-plane and especially clear at low reverberation times. Several same-half-plane conditions show SIR improvements around 9–15 dB (Wang et al., 2018).

In source-image estimation, BSS-Eval-style metrics directly interact with post-separation scaling. A generalized minimum distortion principle (GMDP) replaces the classical squared-error or Gaussian residual model with a mixed-norm residual spectrogram model and optimizes it by majorization-minimization, yielding an iteratively reweighted least-squares update. The reported consequence is that source-image estimation can improve BSS-Eval-style SIR by about 1–2 dB with no decrease in SDR, and up to 2 dB improvement in separation is reported with no increase in distortion. This makes clear that BSS-Eval scores do not merely summarize a separator’s output; they can also be affected by the source-image reconstruction step applied after separation (Scheibler, 2020).

Another nonstationary BSS line uses SIR and SDR in the same sense as standard BSS evaluation even when the paper does not explicitly mention the full BSS-Eval framework by name. In JEFAS-BSS, evaluation combines SIR and SDR with the Amari index, thereby separating waveform-quality assessment from demixing-matrix accuracy. On the synthetic example reported there, JEFAS-BSS outperforms SOBI and p-SOBI on all three criteria, with SIR $\hat{\vs}_m$0 dB, SDR $\hat{\vs}_m$1 dB, and Amari index $\hat{\vs}_m$2. This suggests that BSS-Eval-style waveform measures remain informative in settings where matrix-identification metrics alone would be insufficient (Meynard, 2018).

6. Perceptual limitations and the move beyond BSS-Eval in music separation

Recent musical source separation and singing voice separation studies position BSS-Eval as the conventional baseline while arguing that it correlates imperfectly with perceptual judgments. One study states explicitly that recent work shows “BSS-Eval metrics do not correlate well with perceptual audio quality ratings,” and treats listening tests such as MUSHRA or P.808 DCR as the gold standard, albeit too expensive and slow to run at scale. In that work, BSS-Eval comprises SDR, SIR, SAR, and the scale-invariant SI-SDR, SI-SIR, and SI-SAR, but these measures are treated as established reference points rather than as the site of innovation (Bereuter et al., 22 Apr 2026).

The critique becomes sharper for generative singing voice separation. A 2025 study argues that BSS-Eval metrics were originally designed to evaluate linear audio source separation models based on methods such as time-frequency masking, where the estimate–reference relationship is approximately linear. Generative models can instead create nonlinear relationships between separated and reference signals, so the weak correlation between BSS-Eval and human ratings for such models most likely stems from nonlinearities despite possible perceptual similarity. In that study, BSS-Eval and PEASS remain reasonably aligned with degradation mean opinion scores for discriminative models, with most BSS-Eval and PEASS metrics showing roughly $\hat{\vs}_m$3, but become unrepresentative for generative models, where $\hat{\vs}_m$4 (Bereuter et al., 15 Jul 2025).

The principal alternatives explored are intrusive metrics computed in learned representation spaces. One musical source separation study evaluates mean squared error on MERT embeddings and an intrusive per-song Fréchet Audio Distance, $\hat{\vs}_m$5. Using the MERT-v95 model and layer 12, target and separated signals are encoded at 24 kHz; for 5-second chunks, the representation has $\hat{\vs}_m$6 embedding dimensions and $\hat{\vs}_m$7 time frames. The embedding MSE is

$\hat{\vs}_m$8

and the intrusive FAD is

$\hat{\vs}_m$9

Across two independent datasets with MUSHRA and P.808 DCR ratings, these embedding-based metrics correlate more strongly with perceptual ratings than traditional BSS-Eval metrics across all analyzed stem and model types. The strongest correlations are generally observed for vocals; bass stems show lower correlation for all metrics; and for FAD, SRCC can exceed PCC, implying a monotonic but not strictly linear relation to human ratings (Bereuter et al., 22 Apr 2026).

The generative singing voice separation study reaches a closely related conclusion. Intrusive embedding-based metrics using pretrained encoders such as CLAP, MERT-L12, and Music2Latent, along with multi-resolution STFT loss and ViSQOL, outperform BSS-Eval on generative models. Music2Latent MSE achieves the highest discriminative correlation, but is not reliable for generative SVS; MERT-L12 embedding MSE provides the most balanced performance across discriminative and generative model types. The practical recommendation is therefore not to rely on BSS-Eval alone for generative SVS evaluation and, more broadly, not to assume a single one-metric-fits-all score for musical source separation (Bereuter et al., 15 Jul 2025).

Taken together, these results do not imply that BSS-Eval is obsolete. They show instead that BSS-Eval remains the baseline reference and a useful evaluator for conventional mask-based or approximately linear separation models, while its signal-model assumptions are increasingly mismatched to perceptual assessment and to nonlinear generative separators. A plausible implication is that future evaluation practice will remain hybrid: BSS-Eval for controlled, reference-based decomposition of interference, distortion, and artifacts, and representation-based or listening-test-grounded metrics when perceptual faithfulness is the primary target.