Skip to content

feat: add Projects feature for isolated document + eval workspaces #2

Description

@paramjeetn

Problem

Currently all ingested documents and eval runs share a single global namespace. If a user adds multiple datasets, every eval run searches across all documents — there is no way to scope a test to a specific set of documents.

Goal

Introduce a Projects concept that gives users isolated workspaces. Each project has its own:

  • Document collection (uploaded files)
  • Ground truth / test dataset (question-answer pairs)
  • Eval runs (scoped only to that project's documents)
  • RAG config settings

User Story

As a user, I want to create a Project called "ML Docs Evaluation", upload specific documents to it, add my ground truth Q&A pairs, run evals, and have those evals only search within that project's documents — not across everything I've ever ingested.

Detailed Plan

See docs/plans/projects-feature.md for the full implementation plan.

Scope

  • Backend: new projects table, all existing tables get project_id FK, Qdrant payloads tagged with project_id, all API routes scoped
  • Frontend: Projects dashboard, project switcher, per-project documents + evals + ground truth management
  • Migration: existing data moved into a default project

Labels

  • enhancement

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions