Skip to content

Add reproducible evaluation pipeline and stronger baselines #1

Description

@mkarson1997

Goal

Upgrade the project from a course-style notebook into a reproducible ML case study.

Completed portfolio/reproducibility foundation

  • Added MODEL_CARD.md with intended use, out-of-scope clinical use and limitations.
  • Added EXPERIMENT_PROTOCOL.md covering data provenance, deterministic splits, seeds, preprocessing, training configuration, required metrics and baseline comparisons.
  • Added explicit MIT licensing for project-authored material.
  • Added THIRD_PARTY_NOTICES.md separating dataset/framework rights from project licensing.
  • Added PR checklist for reproducibility/responsible-use review.
  • Added dependency-free scripts/validate_project.py.
  • Added GitHub Actions ML Portfolio Quality workflow.
  • CI validates that the notebook is readable/structured and that required model/reproducibility/security/license artifacts exist.
  • Latest quality workflow is green.
  • README now exposes the quality badge and local validation command.

Remaining experimental work

  • Replace the tiny 16-image validation design with a stronger deterministic split.
  • Record fixed seeds and exact environment/dependency versions for the next run.
  • Report precision, recall/sensitivity, F1 and ROC-AUC alongside accuracy.
  • Add a simple baseline for comparison.
  • Add at least one transfer-learning baseline such as MobileNetV2 or DenseNet.
  • Export/record metrics consistently so runs can be compared.
  • Add confidence intervals and/or calibration analysis where practical.

Acceptance criteria

The issue should close only after a new reproducible experiment follows EXPERIMENT_PROTOCOL.md and the README presents an honest baseline-vs-model comparison without implying clinical readiness.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions