refactor: extract eval infrastructure to aikit-evals crate#156
Merged
Conversation
Removes the 6 generic eval modules (suite, config, runner, artifacts, checks, trace) from fastskill-evals and re-exports them from the new aikit-evals crate. fastskill-evals is now a thin adapter that only keeps config_adapter.rs (reads skill-project.toml). Also fixes a pre-existing test isolation bug: three init tests were calling set_current_dir without DIR_MUTEX or a DirGuard, leaving the process CWD pointing at a deleted TempDir and causing flaky failures in unrelated add tests. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Now that aikit PR #33 is merged, point fastskill-evals at the published crate via git instead of a local workspace path. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
suite,config,runner,artifacts,checks,trace) fromfastskill-evalsinto a newaikit-evalscrate in goaikit/aikit (merged via PR feat: increase test coverage and improve CLI test suite #33)fastskill-evalsis now a thin adapter: re-exports everything fromaikit-evalsand keeps onlyconfig_adapter.rs(reads[tool.fastskill.eval]fromskill-project.toml)commands::add— three tests ininit.rswere missing theDIR_MUTEXguard, causingset_current_dirraces across threadsDependency change
Test plan
cargo build --workspace— clean build against git depcargo test -p fastskill-evals— 2/2 pass🤖 Generated with Claude Code