Skip to content

#409 Parallel file processing with rayon #413

Closed

Description

@ajianaz

Problem

File discovery, parsing, and DB write is fully sequential. No use of multi-core.

Location

src/index/mod.rs:222-258 -- for entry in walker { ... index_file() }

Solution

  1. Add rayon dependency
  2. Collect files into Vec, then par_iter() for parsing
  3. Parse files in parallel (CPU-bound)
  4. Write results via channel to single writer thread

Expected Impact

2-4x faster indexing on multi-core.

Dependency

Blocked by #404, #405, #404b.

Activity

  1. ajianaz commented on Jul 28, 2026

    @ajianaz
    CollaboratorAuthor

    Superseded. #413 was about parallel file processing — already resolved by PR #422 (Rayon). Parent tracker no longer needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions