Skip to content
azrabano23Public

About

Neuro-physiological cognitive-state guardian for aviation safety: multimodal biosignal ML (EEG/ECG/respiration/EDA) that flags dangerous pilot cognitive states in real time. Honest leave-one-subject-out evaluation.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

8 Commits

Folders and files

Repository files navigation

VIGIL

A neuro-physiological cognitive-state guardian for aviation safety.

Most loss-of-control aviation accidents trace back to a loss of airplane-state awareness — a pilot who is distracted, fixated, or startled at the wrong moment. VIGIL is a defensive ML pipeline that estimates a pilot's cognitive state from multimodal biosignals (EEG, ECG, respiration, electrodermal activity) so that dangerous states can be flagged in real time, before they become accidents.

It sits at the intersection of aerospace · neuroscience · medicine, and it is deliberately defensive: the output is a cockpit alert that protects the operator, not a weapon system.

Status: working end-to-end on real human EEG (36 subjects, PhysioNet EEGMAT) and on a physiologically grounded simulator, with drop-in adapters for aviation-specific datasets. Honest leave-one-subject-out evaluation throughout. Tests passing.


The cognitive states

VIGIL classifies four states (the dangerous three are drawn directly from NASA's aviation-safety taxonomy):

State Why it kills Signature VIGIL looks for
Baseline — (calm, nominal) balanced spectra, stable HR
Channelized attention target fixation → flying a working aircraft into terrain frontal-midline theta ↑, posterior alpha ↓
Diverted attention task-switching, missed instruments fluctuating beta ↑, irregular respiration
Startle / surprise freeze / overcorrection after a jolt broadband EEG transient, HRV collapse, EDA arousal

The headline: honest evaluation, not leaked accuracy

The central methodological point of this project: in biosignal classification, how you split the data decides whether your number means anything.

If epochs from the same person land in both train and test (a random/stratified k-fold over epochs), the model memorizes each subject's physiological "fingerprint" and the score is wildly optimistic. The only deployment-relevant number is leave-one-subject-out (LOSO) — train on some pilots, test on a pilot the model has never seen.

VIGIL computes both and reports the gap, so the leakage is visible instead of hidden:

chance balanced-acc       : 0.250
LOSO balanced-acc (HONEST): 0.958     <- the number that counts
within-subject (LEAKY)    : 1.000     <- what you'd wrongly report
>> leakage inflation      : +0.042 balanced-acc if you split wrong

safety-critical recall (LOSO) — fraction of dangerous episodes caught:
  channelized_attention    1.000
  diverted_attention       0.979
  startle_surprise         0.875

(Numbers above are on the synthetic simulator — see "Honesty" below.)

LOSO confusion matrix

…and on REAL human brains

The same pipeline and the same honest split, run on PhysioNet EEGMAT — 36 real subjects, 19-channel 10-20 EEG, rest vs. a mental-arithmetic cognitive-load task (./run.sh eegmat):

subjects                  : 36   (balanced 540 / 540 epochs)
chance balanced-acc       : 0.500
LOSO balanced-acc (HONEST): 0.625     AUROC 0.678     <- subject-independent
within-subject (LEAKY)    : 0.817
>> leakage inflation      : +0.192 balanced-acc if you split wrong

EEGMAT real-data confusion

This is the part that matters. On real brains the honest cross-subject ceiling is modest (0.625) — and a naive within-subject split would have you report 0.817. That +0.19 gap is the whole thesis, and it is ~5× larger on real data than on the clean simulator (+0.04). A model that looks great in a paper and fails on the next pilot is the failure mode VIGIL is built to expose. The numbers are deliberately un-inflated: subject-independent EEG decoding is genuinely hard, and pretending otherwise is how unsafe systems ship.


How it works

 raw biosignals          neurophysiological features            model              decision
 ──────────────          ──────────────────────────            ─────              ────────
 EEG  (8 ch, 256 Hz) ─┐   EEG  band powers (δ θ α β γ),       HistGradient-      per-epoch state
 ECG  ───────────────┤   theta/alpha & beta/alpha ratios,    Boosting on   ──►   + debounced
 RESP ───────────────┼─► frontal-theta / posterior-alpha,    69 features         real-time alert
 EDA  ───────────────┘   HRV (SDNN/RMSSD/pNN50), resp rate                       (vigil.stream)
                          & irregularity, EDA SCL/SCR
  • Handcrafted, auditable features rather than raw-signal black boxes. With only a handful of subjects, physiologically meaningful features generalize across people far better — and when the output is a safety alert, you want to know why it fired. (A compact 1-D CNN baseline is included in vigil/models.py for comparison.)
  • Permutation importance exposes which signals drive each call.
  • Streaming guardian with temporal debouncing — a single noisy frame never trips the alarm; a sustained dangerous episode does.

Quickstart

git clone https://github.com/azrabano23/vigil
cd vigil
./run.sh demo      # simulate → features → honest LOSO vs leaky eval → plots
./run.sh eegmat    # REAL DATA: download PhysioNet EEGMAT (36 subj) + honest eval
./run.sh stream    # live-monitor a held-out pilot, with alerts
./run.sh test      # deterministic end-to-end tests
./run.sh train     # feature-importance ranking

No GPU required. run.sh uses your current Python if the deps are importable, otherwise it builds a .venv and installs requirements.txt.


Data sources (reputable, with licenses)

VIGIL is dataset-agnostic. Internally everything is a list of epochs plus a label array and a subject-id array. Full provenance in DATA.md.

  1. PhysioNet EEGMAT (real, auto-downloaded by ./run.sh eegmat) — 36 real subjects, 19-ch 10-20 EEG, rest vs. mental-arithmetic cognitive load. ODC Public Domain Dedication (fully open, no auth). The real-data backbone. Loader: vigil.realdata.load_eegmat. (Zyma et al., Data 2019.)
  2. Synthetic simulator (default, no download) — multimodal biosignals built from the mechanisms each signal reflects, with state modulations from the aviation-neuroergonomics literature. Exercises the full multimodal pipeline and demonstrates the methodology — not a clinical claim.
  3. NASA — "Reducing Commercial Aviation Fatalities" (Kaggle, 2019) — 18 pilots, 22-ch EEG + ECG + respiration + GSR at 256 Hz, labeled with the dangerous states. Released by NASA with Booz Allen Hamilton. Used under Kaggle's competition terms; this repo does not redistribute it. Adapter: vigil.data.load_nasa_rcaf (needs a Kaggle account).
  4. UAB — Flight-Deck Mental-Workload EEG — CC-BY 4.0, Universitat Autònoma de Barcelona, published in Sensors. Adapter: vigil.data.load_uab.

Honesty

  • Two regimes, clearly separated. The simulator numbers (LOSO 0.958) show the pipeline and methodology are sound on clean signal. The real-data numbers (EEGMAT, LOSO 0.625 / AUROC 0.678) are the honest, un-inflated truth about subject-independent EEG decoding — and they are reported next to the leaky 0.817 so the inflation is impossible to miss. The simulator is physiologically grounded but is not real pilots; treat its metrics as a sanity check, not an operational claim.
  • The leakage gap is the contribution, not the peak accuracy. A model that scores 1.000 within-subject and collapses across subjects is worthless in a cockpit; VIGIL is built to surface exactly that failure.
  • Not a medical device. Research / educational artifact.

Project layout

vigil/
  simulate.py   physiologically grounded multimodal biosignal simulator
  features.py   EEG band power, HRV, respiration, EDA feature extraction
  data.py       loaders + adapters (synthetic, NASA RCAF, UAB CC-BY)
  realdata.py   PhysioNet EEGMAT real-data loader + honest binary eval
  models.py     HistGradientBoosting (primary) + optional 1-D CNN
  evaluate.py   LOSO vs leaky CV, safety-critical recall, confusion plot
  stream.py     real-time debounced alerting on a held-out pilot
  cli.py        `python -m vigil {demo,eegmat,stream,train}`
tests/          deterministic end-to-end tests
DATA.md         data provenance + licensing

License

Code: MIT. Datasets retain their own licenses (see DATA.md).

About

Neuro-physiological cognitive-state guardian for aviation safety: multimodal biosignal ML (EEG/ECG/respiration/EDA) that flags dangerous pilot cognitive states in real time. Honest leave-one-subject-out evaluation.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages