Skip to content

Feature Request: Shared buffer pool for multiple clients #1095

Description

@danielchang-Z

Title: Feature Request: Shared buffer pool for multiple clients

Body:
Currently each chpool.Client holds a dedicated buffer (~100 MiB), which causes very high memory usage (10+ GiB) when maintaining many clients for multi-node inserts. It would be helpful to allow a shared buffer pool (e.g. via sync.Pool or a custom provider) so buffers can be reused across clients.

Repro steps:

for i := 0; i < 20; i++ {
    pool, _ := chpool.New(chpool.Options{ /* connect to node[i] */ })
    // Each pool allocates its own ~100 MiB buffer
}

With dozens of clients, total memory usage grows significantly.

Expected behavior:
Provide an option to configure a shared buffer provider so multiple clients can reuse the same pool of buffers, reducing overall memory overhead in multi-node scenarios.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions