Context
mem/sakana.py currently calls the Weaviate writer once per log event. Under sustained traffic (~100 logs/s) this creates high HTTP overhead.
Proposal
- Buffer events in memory for N items or ≤ 50 ms. * Use
client.batch.add_data_object to send one batch to Weaviate. * Keep the public API await write(event) unchanged.
Acceptance
Criteria - Integration test posts 100 logs → ≥ 3× faster wall time. - Batch size/delay tunable via env vars SAKANA_BATCH_SIZE, SAKANA_BATCH_MS. - All events still appear in class MemoryEvent.
Code reference
See TODO in mem/sakana.py.
Context
mem/sakana.pycurrently calls the Weaviate writer once per log event. Under sustained traffic (~100 logs/s) this creates high HTTP overhead.Proposal
client.batch.add_data_objectto send one batch to Weaviate. * Keep the public APIawait write(event)unchanged.Acceptance
Criteria - Integration test posts 100 logs → ≥ 3× faster wall time. - Batch size/delay tunable via env vars
SAKANA_BATCH_SIZE,SAKANA_BATCH_MS. - All events still appear in class MemoryEvent.Code reference
See TODO in
mem/sakana.py.