Yiming Shen

Yiming Shen

Statistical Machine Learning · Model Evaluation · EEG & BCI · Research Software
Ph.D. in Computational Sciences

Extended Research & Software CV · 4 pages

Practical Guide: Distribution Distance Metrics for EEG Domain Adaptation

4 minute read

Published:

This note summarizes three continuous-distribution distances that I repeatedly use to quantify source-target shift in EEG domain adaptation workflows.

Why these three?

In practice, I track:

  • Wasserstein distance for geometric transport cost.
  • MMD (with RBF kernel) for distribution mismatch in RKHS.
  • Energy distance for a distance-based discrepancy that is often stable in high dimensions.

All are interpreted as smaller is better, with 0 meaning perfect overlap.

Core Definitions

For source distribution \(\mu_S\) and target distribution \(\mu_T\):

\[W_1(\mu_S,\mu_T)=\inf_{\gamma\in\Pi(\mu_S,\mu_T)} \int \|x-y\|\,d\gamma(x,y).\]

For samples \({x_i}{i=1}^m\), \({y_j}{j=1}^n\), MMD is:

\[\mathrm{MMD}^2 =\frac{1}{m^2}\sum_{i,j} k_\sigma(x_i,x_j) +\frac{1}{n^2}\sum_{i,j} k_\sigma(y_i,y_j) -\frac{2}{mn}\sum_{i,j} k_\sigma(x_i,y_j).\]

Energy distance:

\[D_E=\sqrt{2\,\mathbb{E}\|X-Y\|-\mathbb{E}\|X-X'\|-\mathbb{E}\|Y-Y'\|}.\]

Typical Workflow

  1. Standardize both domains in the same feature space (z-score or whitened feature space).
  2. Compute all three metrics before adaptation.
  3. Apply adaptation (e.g., CORAL, OT, SA, TCA, ART/PT).
  4. Recompute all three metrics after adaptation.
  5. Report both distance reduction and target-domain accuracy together.

Interpretation Heuristic

In my EEG pipelines (standardized feature space), Wasserstein values are often interpreted as:

  • 0-2: light shift, transfer may already be acceptable.
  • 2-10: moderate shift, adaptation is usually beneficial.
  • >10: strong shift, naive transfer often fails.

These are heuristics, not universal thresholds. The same numeric value can mean different things under different feature constructions.

Feature dimension is one concrete reason. Two samples from the same distribution still give \(W_1 > 0\), and in a standardized feature space this floor grows with the number of features. Correlated features raise it less than independent ones. Adding samples barely lowers it once the dimension is high. So compare against a same-distribution baseline before reading the thresholds.

W1 between two samples, against feature dimension d (log scale)
Energy distance, same distribution
MMD2, σ = √d, same distribution
Same distribution: W1…
Shift 0.5 per dimension: W1…
Same distribution: energy distance…
Same distribution: MMD2, σ = √d…
Synthetic data, computed in your browser from fixed seeds. Each teal point compares two independent samples of n points from the same distribution, N(0, Id). These features are uncorrelated; the labels under 6, 22 and 253 only give the size of common EEG feature sets. Real features are correlated, which lowers their effective dimension and typically their floor, so compute the floor on your own features rather than reading it off this chart. The amber line uses the same samples with the second one shifted by 0.5 in every coordinate, so its population W1 is exactly 0.5√d (dashed). W1 is exact: an optimal one-to-one assignment with Euclidean cost (Hungarian algorithm), which for two equal-size, equally weighted samples is the exact EMD of the definition above. The bands are this post's heuristic for the author's EEG pipelines; the figure does not know which features, dimension or sample size they were set on, so it shows why to calibrate, not whether the bands are right. Split-half baseline: shuffle one sample, split it into halves of n/2, and compute W1 between them. The two smaller panels use V-statistics like the definitions above, each on its own axis; σ = √d is a choice made for this figure. With only four seeds their lines are noisy, so each panel also shows the exact expected value at the current n (dashed); for energy distance this is √E[ED²], which slightly overestimates the expected distance. Their same-distribution floors shrink as n grows, while the W1 floor shrinks much more slowly once d is large. The energy-distance floor still rises with d, but far less than the W1 floor.

A cheap baseline comes from the source data itself: shuffle it, split it in half, and compute \(W_1\) between the halves. With \(m\) source samples, each half has only \(m/2\), so compare it with a source-target value computed on \(m/2\) samples per domain.

Practical Takeaway

Do not trust a single metric. If one decreases while the others increase, it usually indicates a geometry mismatch (for example, covariance alignment improved but support mismatch remains). I use the three together as a compact diagnostic panel before model selection.

You May Also Enjoy