Skip to content

refactor: extract eval infrastructure to aikit-evals crate#156

Merged
aroff merged 2 commits into
mainfrom
refactor/extract-evals-to-aikit
May 13, 2026
Merged

refactor: extract eval infrastructure to aikit-evals crate#156
aroff merged 2 commits into
mainfrom
refactor/extract-evals-to-aikit

Conversation

@aroff

@aroff aroff commented May 13, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Moved the 6 generic eval modules (suite, config, runner, artifacts, checks, trace) from fastskill-evals into a new aikit-evals crate in goaikit/aikit (merged via PR feat: increase test coverage and improve CLI test suite #33)
  • fastskill-evals is now a thin adapter: re-exports everything from aikit-evals and keeps only config_adapter.rs (reads [tool.fastskill.eval] from skill-project.toml)
  • Fixed pre-existing flaky test failures in commands::add — three tests in init.rs were missing the DIR_MUTEX guard, causing set_current_dir races across threads

Dependency change

# Before (local path, dev-only)
aikit-evals = { path = "../../goaikit/aikit/aikit-evals" }

# After (git reference, aikit PR #33 merged at a22a619f)
aikit-evals = { git = "https://github.com/goaikit/aikit", branch = "main" }

Test plan

  • cargo build --workspace — clean build against git dep
  • cargo test -p fastskill-evals — 2/2 pass
  • Pre-push hook: full 336-test suite passes (0 failures)
  • Clippy clean across all crates

🤖 Generated with Claude Code

aroff and others added 2 commits May 13, 2026 21:08
Removes the 6 generic eval modules (suite, config, runner, artifacts,
checks, trace) from fastskill-evals and re-exports them from the new
aikit-evals crate. fastskill-evals is now a thin adapter that only
keeps config_adapter.rs (reads skill-project.toml).

Also fixes a pre-existing test isolation bug: three init tests were
calling set_current_dir without DIR_MUTEX or a DirGuard, leaving the
process CWD pointing at a deleted TempDir and causing flaky failures
in unrelated add tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Now that aikit PR #33 is merged, point fastskill-evals at the published
crate via git instead of a local workspace path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@aroff
aroff merged commit ee5bfa6 into main May 13, 2026
11 checks passed
@aroff
aroff deleted the refactor/extract-evals-to-aikit branch May 13, 2026 21:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant