Skip to content

Request: access to Delta fixture corpus for benchmark participation #2

Description

@petersimmons1972

Hi CueCrux team,

We've been working through the ScoreCrux benchmarks and have completed both practice-info-101 and practice-safety-101 runs (bare + Engram memory arms). We'd like to participate in the Delta benchmark next.

What we've tried

  • GET /api/fixtures/delta → 404
  • GET /api/fixtures → only lists practice-info-101
  • Searched CueCrux org repos (ScoreCrux, Crux, EmbedderCrux, ResearchCrux, TestCrux) — no corpus data found
  • AuditCrux repo (referenced in how-to-run docs) returns 404

What we're trying to do
Run the Delta benchmark (3,346 documents, 2M tokens, 25 architectural decisions + 5 needle facts) with two arms:

  1. Bare baseline (C0) — no memory system
  2. Engram arm — local MCP memory server, ingest corpus via memory_store_batch, answer via memory_recall

Questions

  1. How do we get access to the Delta fixture and corpus?
  2. Is there an alpha/beta access program, or a fixture ID we should use?
  3. Is AuditCrux the right harness for the C0/F1 arms, and if so, is it available?

Submitter: Peter Simmons
Practice runs completed: practice-info-101 and practice-safety-101 (both bare + Engram arms)

Thanks

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions