Web App: ShiftDx
Milestone:
Application: ShiftDx
Title: Drift Diagnostics and Recalibration Dashboard
Language: Python / Streamlit
Authors: Yiming Shen and David Degras
Overview
ShiftDx is the diagnostic and decision layer of the toolkit. It is not a feature library and not a domain-adaptation method library. Instead, it turns fixed-reference cross-session experiments into operational answers: whether a deployed BCI is drifting, how much accuracy is being lost, whether domain adaptation helps, and when retraining is unavoidable.
Toolkit Role
ShiftDx consumes feature, drift, accuracy, and DA outputs and presents them as a decision dashboard.
Feature and DA outputs -> ShiftDx -> drift diagnosis and adapt-vs-retrain decision
Main Capabilities
- Dataset overview and per-subject drift trajectories.
- Five drift metrics: MMD, Energy, Wasserstein, Mahalanobis, and Euclidean distance.
- Claim-explorer pages for drift-loss association, DA decomposition, retraining gap, and feature robustness.
- Subject-level deep dives across No DA, DA, and Retrain strategies.
- DA Lab pages for live synthetic shift experiments, method sweeps, and Page-Hinkley drift detection.
- Data-quality and statistical-report pages for checking whether dashboard claims are supported by the data.
Protocol
The app follows a fixed-reference deploy-and-monitor protocol: the feature extractor and classifier are fit on session 0 and then applied unchanged to later sessions. This makes the dashboard closer to a real deployment setting than a fully re-tuned retrospective benchmark.