perf(orchestrator): optimize AI calls for rate limits and token usage

README.md

@@ -75,7 +75,20 @@ Go to your repo → Settings → Secrets and variables → Actions:
 
 > `GITHUB_TOKEN` is automatically provided by GitHub Actions.
 
-#### 3. Done!
+> `GITHUB_TOKEN` is automatically provided by GitHub Actions.
+
+#### 3. (Optional) Tune AI Performance
+
+Add these optional secrets or variables to customize the review process:
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `AI_CONCURRENCY` | `1` | Number of files to review in parallel. Set to `1` for low-tier (10k TPM) keys. Increase to `3-5` for high-tier. |
+| `AI_MAX_RETRIES` | `3` | Number of times to retry a failed/rate-limited AI call. |
+| `AI_RETRY_DELAY` | `2000` | Initial delay in ms before retrying (exponential backoff applied). |
+| `AI_BACKOFF_FACTOR` | `2` | Multiplier for delay after each retry. |
+
+#### 4. Done!
 
 Open a PR and watch J Star review it.
 

docs/HISTORY.md

@@ -172,3 +172,17 @@ Fixed critical runtime safety issues identified by the bot's own self-audit:
 | `[latest]` | Fix deleted file logic and add emoji reactions |
 | `[latest]` | Fix duplicate workflow triggers |
 | `[latest]` | Harden runtime safety (orchestrator.ts) |
+
+---
+
+## Phase 8: Stability under Pressure
+**Date:** Dec 14, 2024
+
+### What Happened
+Stabilized the bot against **Rate Limit Exceeded** crashes when using lower-tier API keys (e.g., Kimi 10k TPM).
+
+### Key Decisions
+1.  **Configurable Concurrency:** Added `AI_CONCURRENCY` (default: 1) to allow users to throttle the bot down to sequential processing or scale it up.
+2.  **Smart Delay:** Introduced an artificial delay between file chunks, but *only* when running in sequential mode. This prioritizes stability for free-tier users while unblocking speed for pro users.
+3.  **Recursion Rewrite:** Fixed a critical bug where the retry logic wasn't awaiting its own recursive calls.
+4.  **Conservative Limits:** Reduced the single-shot token limit from 8000 to 6000 to provide a safer buffer against hard API bursts.

docs/SPAWN_GUIDE.md

@@ -119,10 +119,20 @@ review:
   secrets:
     GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
   with:
+    triage_model: 'openai/gpt-oss-120b'      # Fast/cheap for initial classification
     triage_model: 'openai/gpt-oss-120b'      # Fast/cheap for initial classification
     analyst_model: 'moonshotai/kimi-k2-instruct-0905'  # Powerful for deep review
 ```
 
+### Tuning Performance (Optional)
+
+If you are hitting rate limits (429 errors), you can tune the concurrency via **Secrets** or **Variables**:
+
+- `AI_CONCURRENCY`: Positive integer. Set to `1` (default) for safety, or `3-5` for speed.
+- `AI_MAX_RETRIES`: Non-negative integer. Default `3`.
+- `AI_RETRY_DELAY`: Non-negative integer (ms). Default `2000`.
+- `AI_BACKOFF_FACTOR`: Number >= 1. Default `2`.
+
 ---
 
 ## Manual Review Trigger

docs/features/analyst.md

@@ -11,6 +11,8 @@ After Triage identifies risky files, the Analyst performs a thorough review usin
 
 ## Input
 - The actual diff content of the `files_to_audit`.
+  - **Single-Shot Limit:** < 6000 tokens (approx).
+  - **Chunked:** Used automatically if > 6000 tokens.
 - Project context (`.jstar/architecture.md`, `.jstar/rules.md`).
 - Existing docs inventory (`docs/features/*.md`).
 

docs/features/map-reduce.md

@@ -17,13 +17,15 @@ const fileDiffs = splitDiffByFile(diff);
 ### 2. Map (Review Each File)
 Each file chunk is reviewed independently in parallel batches:
 ```typescript
-const BATCH_SIZE = 3; // Respects RPM limits
+const BATCH_SIZE = parseInt(process.env.AI_CONCURRENCY) || 1; // Default 1 (Sequential)
 for (let i = 0; i < relevantDiffs.length; i += BATCH_SIZE) {
   const batch = relevantDiffs.slice(i, i + BATCH_SIZE);
   const batchResults = await Promise.all(batch.map(fd => reviewFileChunk(...)));
 }
 ```
 
+Configurable via `AI_CONCURRENCY`. Set to `1` for strict rate limits (default), or `3-5` for higher tiers.
+
 ### 3. Reduce (Aggregate)
 All chunk results are combined into a final `JStarReviewResult`:
 - Findings are merged into a single array.