Repository navigation
Closed
Labels
enhancementNew feature or requestNew feature or request
Description
Objective
Replace the rules-only delay score with a reproducible trained-model milestone while retaining baseline-rules-v1 as the benchmark and fallback.
Data source
Use the official U.S. Bureau of Transportation Statistics Airline On-Time Performance data:
- https://www.bts.gov/browse-statistical-products-and-data/bts-publications/airline-service-quality-performance-234-time
- https://transtats.bts.gov/DatabaseInfo.asp?DB_URL=&QO_VQ=EFD
Record the exact months, download date, selected fields, and transformation logic. Do not commit a large raw dataset to Git.
Acceptance criteria
- Define the prediction time and target: arrival delay of at least 15 minutes
- Create a documented data contract and reproducible preprocessing command
- Prevent target leakage by excluding actual arrival/departure outcomes unavailable at prediction time
- Use a chronological train/validation/test split
- Compare a naive majority baseline, logistic regression, and one tree-based model
- Report ROC-AUC, PR-AUC, precision, recall, F1, Brier score, and calibration
- Include error slices by carrier, origin airport, route volume, and time period
- Choose the operating threshold from a documented operational cost assumption
- Save versioned model metadata without committing unsafe or opaque serialized artifacts
- Publish a model card with data limitations, intended use, non-goals, and failure modes
- Add unit tests for preprocessing and an integration test for model-backed prediction
- Keep
baseline-rules-v1available as a transparent fallback
Recruiter evidence
The README must show a compact comparison table, one calibration visualization, one error-slice finding, the exact reproduction command, and an honest explanation of what improved and what did not.
Out of scope
- claiming real-time airline production readiness
- claiming causality
- using post-event fields that leak the outcome
- adding an LLM before the predictive model and evaluation are credible
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request
Activity
mitulpatel123 commentedon Aug 8, 2026
The first milestone slice is now in draft PR #3: a fixed T-24h prediction contract, explicit leakage exclusions, chronological evaluation split, calibration/error-slice requirements, and artifact-safety rules. Both CI runs pass.
mitulpatel123 commentedon Aug 8, 2026
Completed in PR #4. The milestone now includes official BTS data lineage, a 90,000-row chronological evaluation, logistic and fixed-budget tree baselines, validation-only threshold selection, calibration, error slices, a portable JSON model, a model card, API integration, and passing CI.