---
title: 'Stream-SW: Streaming Sliced Wasserstein'
url: https://www.emergentmind.com/topics/streaming-sliced-wasserstein-stream-sw
type: topic
---

# Stream-SW: Streaming Sliced Wasserstein

Streaming Sliced Wasserstein (Stream-SW) is a computational framework for estimating the sliced Wasserstein (SW) distance between probability distributions when samples arrive in a streaming fashion. It builds on quantile sketching techniques for 1D Wasserstein computation and extends them via randomized projections to provide a memory-efficient, single-pass algorithm for high-dimensional optimal transport problems. Stream-SW offers theoretical guarantees on accuracy and resource consumption, and demonstrates marked advantages over random subsampling approaches in a variety of empirical settings.

## 1. Streaming 1D-Wasserstein Distance via Quantile Sketches

Sliced Wasserstein methods reduce a $d$-dimensional optimal transport problem to a collection of one-dimensional projection problems. The 1D $p$-Wasserstein distance between empirical measures 
$$\mu_n = \frac{1}{n}\sum_{i=1}^n \delta_{x_i}, \quad \nu_m = \frac{1}{m}\sum_{j=1}^m \delta_{y_j}$$
admits the closed form
$$
W_p^p(\mu_n, \nu_m) = \int_0^1 |F^{-1}_{\mu_n}(q) - F^{-1}_{\nu_m}(q)|^p\, dq,
$$
where $F^{-1}_{\mu_n}$ denotes the quantile function. 

In a streaming context, storage of all samples is not feasible. Instead, Stream-SW maintains a *quantile sketch* $S_{\mu_n,k}$ of fixed size $k$, supporting approximate quantile queries $Q(q; S)$ such that $|Q(q; S)-F^{-1}_{\hat\mu}(q)| \le \epsilon n C$, with $C$ the maximal sample gap. The KKL-sketch of Karnin–Lang–Liberty supports one-pass updates with $O((1/\epsilon)\sqrt{\log(1/\delta)} + \log(n/k))$ memory.

Given two such sketches, the streaming 1D-Wasserstein estimator is
$$
\widetilde W_p^p(\mu_n, \nu_m; S_{\mu_n,k_1}, S_{\nu_m,k_2}) = \int_0^1 |Q(q; S_{\mu_n, k_1}) - Q(q; S_{\nu_m, k_2})|^p dq.
$$
For $p=1$, this reduces to an integral over the absolute difference of quantile queries. Sketches are incrementally updated in $O(\log(n/k))$ amortized time per sample.

## 2. Streaming Sliced Wasserstein Algorithm

The sliced Wasserstein distance of order $p$ for $d$-dimensional measures $\mu$ and $\nu$ is
$$
SW_p^p(\mu, \nu) = \mathbb{E}_{\theta \sim \mathcal{U}(\mathbb{S}^{d-1})} [W_p^p(\theta \sh \mu,\, \theta \sh \nu)],
$$
where $\mathbb{S}^{d-1}$ is the unit sphere and $\theta \sh \mu$ is the projection of $\mu$ onto direction $\theta$.

*Streaming Sliced Wasserstein (Stream-SW)* replaces each $W_p$ with $\widetilde W_p$ from quantile sketches. The algorithm proceeds as follows:

- Select $L$ projection directions $\theta_1, ..., \theta_L \in \mathbb{S}^{d-1}$.
- For each direction $\ell=1,...,L$ and each incoming $x_i$ from $\mu$'s stream, compute $\theta_\ell^\top x_i$ and update $S_{\theta_\ell \sh \mu, k_1}$. Similarly for $\nu$.
- At any point, estimate the SW distance by
  $$
  \widehat{\widetilde{SW}_p^p}(\mu_n, \nu_m; k_1, k_2, L) = \frac{1}{L} \sum_{\ell=1}^L \int_0^1 |Q(q; S_{\theta_\ell \sh \mu, k_1}) - Q(q; S_{\theta_\ell \sh \nu, k_2})|^p dq.
  $$

Stream-SW thereby enables a single-pass, memory-bounded estimate of the SW distance at any time.

## 3. Theoretical Guarantees and Complexity

Stream-SW provides explicit, nonasymptotic bounds on both memory usage and approximation error:

- **Streaming 1DW error:** If the supports have diameter $R$ and sketch precisions $\epsilon_1,\epsilon_2$, the error satisfies
  $$
  |\widetilde W_p^p(\mu_n, \nu_m) - W_p^p(\mu_n, \nu_m)| \le p R^{p-1} (\epsilon_1 n C_1 + \epsilon_2 m C_2).
  $$
- **SW population-level error:** For i.i.d. samples from $\mu, \nu$, the expected error in SW is
  $$
  \mathbb{E} |\widetilde{SW}_p^p(\mu_n, \nu_m) - SW_p^p(\mu, \nu)| \le C_{p,R} \left(\alpha(d, n, m) + \epsilon_1 n + \epsilon_2 m\right)
  $$
  with $\alpha(d,n,m) = \sqrt{d+1}\left(\sqrt{\frac{\log n}{n}} + \sqrt{\frac{\log m}{m}}\right)$.

- **Monte Carlo error in $L$ projections:**
  $$
  \mathbb{E}|\widehat{\widetilde{SW}_p^p} - \widetilde{SW}_p^p| \le \frac{\operatorname{Var}^{1/2}[\,\widetilde{W}_p^p(\theta \sh \mu_n, \theta \sh \nu_m)\,]}{\sqrt L}.
  $$

- **Memory and computational complexity:** Each KKL-sketch uses $k=O((1/\epsilon)\sqrt{\log(1/\delta)})$ space. With $2L$ sketches and direction storage, total space is $O(Lk + L\log(n/k) + Ld)$; per-sample update is $O(L\log(n/k) + Ld)$.

Stream-SW thus achieves $O(n^{-1/2} + k^{-1} + L^{-1/2})$ error rates with memory and time scalable in $L$ and $k$.

## 4. Proof Outline of Approximation Bounds

The approximation guarantees derive from three elements:

- **Quantile approximation:** The sketch replaces the exact quantile function; a Taylor or Hölder expansion delivers a pointwise error proportional to the sketch precision and window width ($O(\epsilon_1 n C_1 + \epsilon_2 m C_2)$).
- **Projection averaging:** The error is averaged over random $\theta$, yielding the same order bound for the SW aggregate.
- **Sampling error decomposition:** The estimator's discrepancy with population SW separates into (i) sketching error and (ii) statistical sampling error, with the latter analyzed via empirical process (VC) bounds over half-spaces $\{x : \theta^\top x \leq z\}$.
- **Monte Carlo in $L$:** Standard MC variance bound applies for the projection average, yielding $O(1/\sqrt L)$ behavior.

## 5. Empirical Evaluation

Stream-SW demonstrates favorable empirical properties across several domains:

| Task                       | Key Finding                                               | Comparison                    |
| -------------------------- | -------------------------------------------------------- | ----------------------------- |
| Mixtures of Gaussians      | 2×–10× lower error than subsample-SW at same $k$         | 10×–100× fewer points retained|
| Point-cloud classification | Stream-SW($k=20$) achieves 76.3–77.7% vs full SW 77.3–77.7% accuracy; subsample-SW ($k=20$): 67.7–68.0% | ModelNet10, KNN ($K=5$)       |
| Gradient flows             | Faster convergence: $W_2^2\approx 26.9$ vs subsample-SW 31.1 at step 1000; only Stream-SW converges in 5000 steps | Euler–Maruyama, 1000 particles|
| Change-point detection     | Detection delay reduced to 10–32 frames (SW sliding window: 49–100) | MSRC-12 Kinect                |

Stream-SW thus attains higher accuracy or faster convergence at fixed memory compared to uniform random subsampling approaches, particularly under severe memory bottlenecks.

## 6. Implementation, Tuning, and Extensions

Key practical considerations when deploying Stream-SW include:

- **Choosing number of projections $L$ and sketch size $k$:** Increasing $L$ reduces MC error as $O(1/\sqrt L)$, whereas increasing $k$ strengthens quantile accuracy as $O(1/k)$. Total memory and per-sample update cost scale linearly with $L$ and logarithmically with $n/k$.
- **Projection schemes:** Replace standard MC by quasi-MC sequences (e.g., Sobol, Halton) or optimized directions for sphere integration, as in quasi-Monte Carlo for 3D SW.
- **Handling asymmetric streams:** In scenarios where only one distribution is streaming, maintain the sketch only for the streaming distribution and compare on-the-fly to the fixed other.
- **Extensions:** The approach adapts to generalized sliced OT, spherical or manifold-projected SW, and partial-SW, by substituting the appropriate 1D streaming OT solver.

In sum, Stream-SW is the first *single-pass*, low-memory methodology for sliced Wasserstein distance estimation from sample streams, with rigorous finite-sample and memory–error guarantees, and empirically outperforms random subsampling algorithms under tight resource constraints [2505.06835].

Source: https://www.emergentmind.com/topics/streaming-sliced-wasserstein-stream-sw