You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Storage] Implement batch ingestion API for high-volume RDF datasets #615
Provide an optimized batch ingestion API in core.next.storage to ingest high-volume RDF graphs without memory exhaustion or excessive garbage collection overhead.
Objectives and Technical Scope
1. Batch Ingestion Interface
• Design a high-throughput batch loading contract (addBatch(Iterable<Statement>) / stream consumer) in core.next.storage.api.
• Implement bulk index population that minimizes pointer re-allocations and tree rebalancing overhead.
2. Index Deferral and Rebuilding
• Support temporary index suspension or bulk insertion modes during massive loads, followed by deterministic index finalization.
Acceptance Criteria
[ ] Bulk loading of 100,000+ statements completes without memory exhaustion or GC stalls.
[ ] All index invariants (SPO, POS, OSP) are fully satisfied and consistent post-ingest.
[ ] Memory footprint per statement during batch loading is measurably optimized compared to one-by-one insertion.
Activity