Pipeline optimization for production scale

[michaeldistel]

Description

As usage scales to production levels, optimize the extraction pipeline for high-throughput scenarios.

Current Performance

Pipeline overhead is currently ~1-2ms per extraction (negligible for single extractions).

Future Considerations

• Connection pooling for LLM APIs
• Response streaming
• Memory optimization for large-scale batch processing
• Metric collection and monitoring

Priority

Low priority currently - pipeline is already very fast. Revisit when scaling needs arise.

Related

• See benchmarks/README.md baseline performance metrics
• Related to Add parallelization support for batch extractions #46 (parallelization)