fix(embeddings): restore provider-backed batches on current main

[seonghobae]

Summary

• restore durable provider-backed embedding batches on current main
• retain current auth and provider-error taxonomy, including typed 413 and structured synthesis failures
• restore and build the pinned native Rust token packer

Replaces the integration path attempted in #857 after #969. #857 remains open pending acceptance of this replacement.

Verification

• focused embedding/registry/cost/provider-error suite: 146 passed
• make test: 2860 passed, 1 skipped
• Rust 1.97.1 cargo build --locked --release passed
• maturin 1.9.4 wheel build passed

All evidence uses synthetic inputs only.

---

Summary by CodeRabbit

• 새로운 기능

  • 원격 임베딩 모델의 비동기 배치 처리, 예약·시작·상태 조회·취소를 지원합니다.
  • 정확한 토큰 계산과 텍스트 분할·샤딩, 임베딩 품질 계산을 제공합니다.
  • 임베딩 모델 자동 검색·라우팅 및 구조화 응답 지원 모델 확인을 제공합니다.

• 개선 사항

  • 확인할 수 없는 사용량은 추정하지 않고 unavailable로 표시하며, 관련 예산은 안전하게 차단합니다.
  • 작업 소유권 상실과 만료를 감지해 중복 결과 게시를 방지합니다.
  • 잘못된 세션 식별자를 거부하고, 사용 불가 모델에 대한 자동 전환과 서버 종료 처리를 안정화했습니다.

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

Provider 임베딩 배치가 durable claim fencing, 예약·시작 분리, 취소, deadline, owner 검증 및 provider usage를 지원합니다. 휴리스틱 토큰 추정은 제거되고 Rust 기반 권위적 카운터와 fail-closed 예산 처리가 추가되었습니다. Structured synthesis, 모델 검색, HTTP 계약 및 Docker 빌드도 갱신되었습니다.

Changes

임베딩 배치 및 토큰 회계

Layer / File(s)	Summary
Provider 배치 실행과 claim fencing contextual_orchestrator/batch_job_registry.py, contextual_orchestrator/batch_routing.py, contextual_orchestrator/cost_router.py, contextual_orchestrator/server.py	Provider 작업이 claim lease, 원자적 terminal 기록, 예약·시작 분리, 취소, deadline, owner 검증 및 서버 종료 처리를 사용합니다.
권위적 토큰 계산과 Rust 패커 contextual_orchestrator/token_counting.py, rust/token_counter/*, Dockerfile	휴리스틱 추정이 제거되었습니다. PostgreSQL 또는 Rust 카운터가 선언된 모델을 처리하고, Rust 확장이 UTF-8 안전 패킹과 수치 검증을 제공합니다.
라우팅, 사용량 및 검증 contextual_orchestrator/orchestrator.py, contextual_orchestrator/__main__.py, tests/*	사용량이 없으면 unavailable로 기록됩니다. 예산은 측정 불가 상태에서 차단됩니다. 임베딩 라우팅과 configured-gateway structured probe가 검증됩니다.

Structured 요청 및 HTTP 계약

Layer / File(s)	Summary
Structured synthesis failover contextual_orchestrator/orchestrator.py, contextual_orchestrator/provider_errors.py, tests/test_model_judge.py, tests/test_chat_response_format_http_honesty.py	가상 structured 요청은 request-scoped 모델 제외 집합을 공유합니다. stale 모델은 같은 endpoint에서 교체될 수 있습니다. 명시적 모델 선택은 대체되지 않습니다.
Session, usage 및 API 응답 contextual_orchestrator/server.py, contextual_orchestrator/api_contract.py, tests/test_openai_passthrough.py, tests/test_streaming.py	session_id를 검증하고 provider payload에서 제외합니다. 사용량은 측정값 또는 null로 반환됩니다. OpenAPI 응답 스키마가 추가되었습니다.
설계 문서와 변경 기록 docs/adr/*, docs/planning/adrs/*, docs/library_research.md, README.md, CHANGELOG.md, contextual_orchestrator/admin.py	Claim fencing, 권위적 토큰 회계, capability probe, 사용량 상태 및 예산 동작이 문서와 관리 화면에 반영되었습니다.

Estimated code review effort: 5 (Critical) | ~120 minutes

Merge Risk: 🟡 Moderate · up to 9f8cd

The PR restores durable provider-backed embedding batches and native token packing, but jobs can remain non-terminal after claim failure, and configured durable deployments can silently lose cross-process recovery when the required client is unavailable. These bounded availability and deployment risks should be fixed or explicitly accepted before merge.

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant CostRoutingCoordinator
  participant ProviderEmbeddingBatchBackend
  participant JobRegistryFactory
  participant EmbeddingProvider
  Client->>CostRoutingCoordinator: 임베딩 배치 예약 및 시작
  CostRoutingCoordinator->>ProviderEmbeddingBatchBackend: reserve 및 start
  ProviderEmbeddingBatchBackend->>JobRegistryFactory: 실행 claim 획득
  ProviderEmbeddingBatchBackend->>EmbeddingProvider: embed_with_usage 호출
  EmbeddingProvider-->>ProviderEmbeddingBatchBackend: 벡터와 provider usage 반환
  ProviderEmbeddingBatchBackend->>JobRegistryFactory: claim 검증 및 terminal 상태 게시
  Client->>CostRoutingCoordinator: 결과 및 usage 조회
  CostRoutingCoordinator-->>Client: 완료, 실패 또는 취소 문서 반환

Loading

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 40.42% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 381 functions across 66 files. (8 skipped…	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	PR 제목은 현재 main에서 provider-backed embedding batches를 복원하는 주요 변경 사항을 정확하고 간결하게 설명합니다.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 40.42% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 381 functions across 66 files. (8 skipped: 7 unsupported, 1 too large.)

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/provider-embedding-current-main

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

Security follow-up after stacked PR #989 merged: exact-head 073b87b restores the shared fail-closed provider-message boundary. Only the exact JSON-object schema diagnostic is canonicalized; other provider prose containing messages/content is suppressed before caller or telemetry exposure. Regression covers natural-language request echo. Focused provider taxonomy/telemetry/reliability suite: 88 passed; Ruff and diff checks passed.

Combined-stack exact-head verification at d81be270: PR #988 endpoint-constrained routing and PR #989 actionable secret-safe provider telemetry are now merged into this feature branch. Full local suite passed: 2954 passed, 2 skipped in 667.23s. Prior review/check evidence for 8c678788 is obsolete; fresh exact-head Security, OpenCode, and Noema evidence is required before main merge.

Resolved the current-head unavailable-aggregate finding in 858a13e. CostLedger.rollup and total now publish null token/cost totals whenever any member is unavailable, while preserving record/unavailable counts, measurement status, and rollup currency metadata. Report sorting also handles mixed known/unknown buckets. Regression covers both all-unavailable and mixed measured/unavailable aggregates. Focused embedding/ledger/provider-boundary suite: 162 passed; Ruff, compileall, and diff check passed.

Current exact-head verification at 073b87b1: the follow-up security commit canonicalizes provider schema diagnostics and defaults provider prose containing request-content field names to non-exportable. Focused telemetry/provider taxonomy: 54 passed; full combined suite: 2954 passed, 2 skipped in 648.68s; diff-check passed. Fresh exact-head hosted Security and review evidence remains required.

Exact current head 858a13ed is stable after the accounting follow-up. Incomplete measurement aggregates now publish null token/cost totals instead of misleading partial sums. Focused cost-ledger suite: 38 passed; full exact-head suite: 2956 passed, 2 skipped in 660.31s; diff-check passed. Hosted exact-head Security/OpenCode/Noema remain required.

Note

This report is out of date. Scroll down for Devin Review's latest report on this PR.

Devin Review found 0 new potential issues.

Exact current head 3f6fce20 verification after stacked PR #990: generated workflow instructions preserve caller authority and multipart content under the provider-neutral orchestration contract. Focused generated-workflow suite: 12 passed; full exact-head suite: 2958 passed, 2 skipped in 650.11s; diff-check passed. Hosted exact-head Security/OpenCode/Noema remain required.

Current-head upstream RCA (3f6fce20ec1a0475f36f775786979b9b7d808ecf):

• The Azure 400 was real: the structured capability probe requested response_format.type=json_object but its message only contained JSON syntax (Return only {"status":"ok"}.), not the literal JSON instruction required by that Azure deployment.
• Current head fixes the request at contextual_orchestrator/__main__.py: Return only this JSON object: {"status":"ok"}.
• Current head also fixes the diagnostic gap: telemetry.operation_failed now records the stable error code, upstream HTTP status, a bounded package-owned schema summary, exact model group, and fallback outcome without exporting provider prose, prompts, API bases, private host IPs, or secrets.
• Exact-tree focused verification: uv run pytest -q tests/test_telemetry.py tests/test_provider_error_taxonomy.py tests/test_chat_model_capability_isolation.py → 77 passed in 3.18s.

Expected safe operator signal for this incident is equivalent to:
error_type=invalid_request_error provider_status=400 error_summary='messages must mention json when response_format is json_object' model_group=gpt-4.1 fallback_outcome=not_attempted.

Devin Review found 3 new potential issues.

🟡 Unpriced embeddings appear free

When provider usage is aggregate-only, price_known remains true for an unpriced model. The completed batch reports zero cost instead of unknown cost.

Suggested change

	total_cost_amount += float(record.cost_amount)
	currency_code = record.currency_code
	aggregate_usage_recorded = True
	total_cost_amount += float(record.cost_amount)
	price_known = price_known and record.price_known
	currency_code = record.currency_code
	aggregate_usage_recorded = True

---

Was this helpful? React with 👍 or 👎 to provide feedback.

🔍 Duplicate ADR identifier

The ADR index now assigns 0005 to two accepted decisions. References to that identifier are ambiguous and need reconciliation.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

📝 Info: Weak local locks stay retained

Each context manager closes over its local lock. Active and waiting claims retain that lock despite the weak dictionary.

---

Was this helpful? React with 👍 or 👎 to provide feedback.