Skip to content

Added 10 new real world questions to the retrieval eval..... - #10

Merged
Krish6115 merged 1 commit into
Krish6115:mainfrom
Rpann:test/add-eval-questions
Nov 3, 2026
Merged

Krish6115 merged 1 commit into
Krish6115:mainfrom
Rpann:test/add-eval-questions

Conversation

@Rpann

@Rpann Rpann commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Adds 10 new real-world questions to kitty/ingest/eval/questions.json covering topics not previously tested:

  • git tag: annotated release tags and pushing tags
  • git worktree: deletion and cleanup
  • git clean -n: preview / dry-run
  • git log --oneline: history graph visualization
  • Commit signing using SSH keys
  • Git aliases configuration
  • .gitattributes format for Git LFS
  • Initializing submodules after clone
  • Advanced .gitignore exception rules
  • GitHub CLI (gh pr create)

Eval Results

  • Evaluated: 62 questions
  • hit@6: 62/62 (100.0%)
  • MRR: 0.841
  • All 216 unit tests and typechecks passing

@vercel

vercel Bot commented Nov 3, 2026

Copy link
Copy Markdown

@Rpann is attempting to deploy a commit to the srkreddy6115-4844's projects Team on Vercel.

A member of the Team first needs to authorize it.

@Krish6115
Krish6115 merged commit 26d3381 into Krish6115:main Nov 3, 2026
1 check failed
@Rpann
Rpann deleted the test/add-eval-questions branch November 4, 2026 13:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants