Skip to content

perf: batch expand operations to reduce DB queries - #7

Merged
Maxim Manuylov (maxmanuylov) merged 3 commits into
jb/v1.6.6from
jb/fix-concurrency-2
Sep 2, 2026
Merged

perf: batch expand operations to reduce DB queries#7
Maxim Manuylov (maxmanuylov) merged 3 commits into
jb/v1.6.6from
jb/fix-concurrency-2

Conversation

@maxmanuylov

Copy link
Copy Markdown
Member

No description provided.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the permission Expand execution to reduce per-entity database queries by batching entity IDs (IN-clause queries) and adds an internal per-operation goroutine concurrency limiter to avoid goroutine explosion during large expands.

Changes:

  • Add functional options to ExpandEngine for concurrency limiting and max batch size, wiring these from service config.
  • Batch Expand’s relation expansion by grouping usersets and querying relationships for multiple entity IDs in a single call.
  • Add/adjust tests to exercise the new batching paths and update version metadata.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
pkg/cmd/serve.go Passes concurrency and batch-size options into the Expand engine using service config values.
internal/info.go Bumps the build/version string.
internal/engines/utils.go Introduces ExpandOption functional options (ExpandConcurrencyLimit, ExpandMaxBatchSize).
internal/engines/expand.go Implements batching for expand operations and adds per-operation goroutine concurrency limiting.
internal/engines/expand_test.go Updates tests for method-based expand operations and adds batching-focused test cases.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread internal/engines/expand.go
Comment thread internal/engines/expand_test.go
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 1, 2026 13:37

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (1)

internal/engines/expand.go:44

  • NewExpandEngine applies options without validating them. If ExpandConcurrencyLimit or ExpandMaxBatchSize are set to 0/negative, expandOperation will deadlock (channel capacity 0) and batching loops using i += engine.maxBatchSize can become infinite. Consider clamping to defaults after applying opts.
func NewExpandEngine(sr storage.SchemaReader, rr storage.DataReader, opts ...ExpandOption) *ExpandEngine {
	engine := &ExpandEngine{
		schemaReader:     sr,
		dataReader:       rr,
		concurrencyLimit: _defaultConcurrencyLimit,
		maxBatchSize:     _defaultMaxBatchSize,
	}
	for _, opt := range opts {
		opt(engine)
	}
	return engine

@maxmanuylov Maxim Manuylov (maxmanuylov) changed the title Use batches for expand operation perf: batch expand operations to reduce DB queries Sep 1, 2026
@maxmanuylov
Maxim Manuylov (maxmanuylov) added this pull request to the merge queue Sep 2, 2026
Merged via the queue into jb/v1.6.6 with commit 93d801b Sep 2, 2026
4 of 9 checks passed
@maxmanuylov
Maxim Manuylov (maxmanuylov) deleted the jb/fix-concurrency-2 branch September 2, 2026 08:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants