Web App: ShiftDx
Milestone:
Application: ShiftDx
Title: Drift Diagnostics and Recalibration Dashboard
Language: Python / Streamlit
Authors: Yiming Shen and David Degras
Overview
ShiftDx examines a practical question: when a model performs worse on later data, how much does adaptation recover, and how much additional benefit remains available from retraining? I built an offline dashboard to inspect these quantities separately in repeated-session EEG experiments, rather than treating every distribution change as evidence of model failure.
Toolkit Role
ShiftDx consumes feature, drift, accuracy, and DA outputs and presents them as a decision dashboard.
Feature and DA outputs -> ShiftDx -> drift diagnosis and adapt-vs-retrain decision
Main Capabilities
- Dataset overview and per-subject drift trajectories.
- Five drift metrics: MMD, Energy, Wasserstein, Mahalanobis, and Euclidean distance.
- Claim-explorer pages for drift-loss association, DA decomposition, retraining gap, and feature robustness.
- Subject-level deep dives across No DA, DA, and Retrain strategies.
- DA Lab pages for interactive synthetic shift experiments, method sweeps, and Page-Hinkley drift-detection demonstrations.
- Data-quality and statistical-report pages for checking whether dashboard claims are supported by the data.
Protocol
The fixed-reference baseline fits the feature extractor and classifier on session 0 and applies them unchanged to later sessions. Separate conditions evaluate domain adaptation and target-session retraining. This distinguishes loss from reusing a fixed model, recovery from adaptation, and the remaining retraining gap.
Interpretation and limits
The dashboard exposes subject trajectories, multiple drift metrics, and statistical summaries so that associations and failures can be inspected rather than assumed. These are offline research analyses; the software is not a validated production monitor, online failure detector, or clinical deployment. The synthetic drift demonstrations are separate from the recorded-EEG evaluation.