Skip to content

BufferPool: make it user-controllable via eval() API #124

@shinaoka

Description

@shinaoka

Summary

Currently BufferPool is always ON, created internally in EinsumNode::eval(), and not controllable by users. For small tensors, jemalloc/tcmalloc tcache is sufficient and the pool adds unnecessary complexity. For large tensors (>4MB), the pool avoids costly mmap/munmap.

Proposal

  1. Make BufferPool public and accept Option<&mut BufferPool> in the eval API:
// With pool (user-managed, reusable across calls)
let mut pool = BufferPool::new();
node.eval_with_pool(&operands, &mut pool)?;

// Without pool (allocator handles everything)
node.eval(&operands)?;
  1. This also enables sharing a single pool across multiple einsum calls.

Benefit

  • Users can benchmark ON vs OFF to determine if the pool helps their workload
  • Pool lifetime is user-controlled (can persist across calls or be dropped)
  • Cleaner separation of concerns

Risk

Low. The pool is already passed as an argument internally (eval_node(..., pool, ...)). This just exposes it at the public API level.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions