OCRKit

Screenshot recognition API for Bastion.

Stack

• Python 3.12+
• FastAPI
• RapidOCR + ONNX Runtime (default)
• OpenCV
• uv

Run

git clone --recurse-submodules git@github.com:OWBastion/ocrkit.git
cd ocrkit
uv sync
uv run uvicorn app.main:app --host 0.0.0.0 --port 8000

For an existing checkout, initialize the private datasets submodule before running tests or training commands:

git submodule update --init --recursive

API

• GET /health
• POST /api/v1/ocr/challenge (multipart/form-data with file, optional debug=true)
• POST /api/v1/ocr/challenge/by-object (application/json with object_key, optional bucket, version_id, debug)

Asynchronous platform jobs

POST /api/v1/ocr/challenge/jobs accepts multipart file and UUID job_id, using Authorization: Bearer <OCRKIT_API_TOKEN>. It returns HTTP 202 {"jobId":"<UUID>","status":"accepted"} after committing the image to a private local SQLite spool; acceptance does not wait for inference. The same ID and image can be submitted again without starting a second recognition. A different image for an existing ID returns 409. Invalid IDs/images are rejected; a full queue returns 503. The synchronous endpoints remain available for smoke checks and training consumers.

One background inference worker shares the engine lock with synchronous requests. Inference and object downloads run outside the ASGI event loop, so health checks and new admissions remain responsive. Queued jobs older than 30 minutes fail with OCR_JOB_EXPIRED before inference starts. This is a queue deadline, not an inference execution timeout: the synchronous engine cannot be safely interrupted. The platform may expire a submission at its own 45-minute deadline and ignore a later callback.

OCRKit posts JSON to the fixed configured OCRKIT_PLATFORM_BASE_URL HTTPS origin at /v1/ocrkit/jobs/{jobId}/result, authenticated by the same service token:

{"contractVersion":"1","result":{"request_id":"<jobId>","ok":true}}

result is the complete existing ChallengeResponse. On recognition failure the body instead contains errorCode: "OCR_RECOGNITION_FAILED"; queued expiration uses errorCode: "OCR_JOB_EXPIRED". Caller-supplied callback URLs are not supported and HTTP redirects are rejected. Any 2xx acknowledges delivery, including 204 for an already completed or stale platform job. Other responses and network failures retry with exponential delay capped at 60 seconds, using a 10-second callback timeout; callback retries reuse the persisted result and do not run OCR again.

OCRKIT_JOBS_DB_PATH defaults to .runtime/jobs.sqlite3. Compose stores it in the persistent ocrkit-jobs volume. Run one Uvicorn process and one service replica per spool; startup takes an exclusive file lock and rejects a second process. Restart resumes persisted images/results. A crash during inference can rerun that unfinished recognition, while committed results are delivered without repeating inference.

The queue accepts at most 100 pending jobs and 10,000 retained IDs. Images are cleared with SQLite secure deletion when recognition finishes; results are cleared on callback acknowledgement. Only the ID/image hash remain for deduplication. All records, including undelivered results, expire 24 hours after acceptance. Keep this volume private: it is transient OCR delivery state, and is not platform business data or training material.

Successful recognition responses retain the data, warnings, and optional debug fields and also include traceable contract metadata:

{
  "schema_version": "1",
  "request_id": "...",
  "engine": "rapidocr",
  "model_version": "builtin",
  "layout_version": "1280x720-v10",
  "ok": true,
  "data": {},
  "fields": {
    "viewer_player": {
      "value": "...",
      "confidence": 0.98,
      "source_roi": ["bottom_left_hero"],
      "normalization": [],
      "status": "ok"
    }
  },
  "warnings": [],
  "quality": {
    "original_size": [2560, 1440],
    "aspect_ratio": 1.7778,
    "layout_confidence": 1.0,
    "cropped": false,
    "blur_score": 0.08,
    "normalized_size": [1280, 720],
    "layout_version": "1280x720-v10",
    "warnings": []
  }
}

quality describes input evidence before normalization. aspect_ratio is rounded to four decimal places; cropped and the related warnings indicate a ratio mismatch against the configured layout and do not make an approval decision. blur_score is a normalized blur risk from 0 to 1, where higher means blurrier.

Layout regions (1280x720-v10 / 1280x800-v5): settlement_card (the completion card Bastion shows for 20 seconds at the top centre after a run), left_panel (stats; reaches the clear-time row of older HUD revisions), run_code_panel, run_code_right_panel, right_panel (map, mode, timers, version), server_load_panel, bottom_left_hero (hero card, the name is its last token), achievement_panel (the stack under the stats block: at most one event, then up to five achievement unlocks; achievements move up when no event is shown, so the ROI is tall and lines are told apart by content), center_banner (completion banner), and ai_watermark (bottom-right generation mark; empty on genuine screenshots). Event/achievement/AI-mark fields are added to the response separately from the geometry.

The service selects the versioned 1280x720-v10 or 1280x800-v5 ROI layout from the source aspect ratio. The 16:10 layout is used for screenshots such as 2560x1600 without marking the input as cropped. 21:9 ultrawide screenshots (about 2.39:1, such as 3440x1440) use the 16:9 layout at the same height scale, with left, right and central ROIs kept at their screen edges or middle, and are not marked as cropped.

Send X-Request-ID to correlate a request; otherwise OCRKit generates one. Each field reports its parsed value, confidence, source ROI, normalization metadata, and status. achievement_panel_text contains raw OCR evidence from the dedicated left-side achievement panel ROI. It may contain the panel's progress and statistics in addition to the challenge text; the platform should match it against its current challenge/title catalog and decide whether the evidence is adopted or grants anything. achievement_titles contains every text candidate detected in the fixed achievement ROI when OCR sees it followed by a visible completion checkmark, in top-to-bottom order. They are raw candidates rather than catalog matches; the platform owns catalog matching and business decisions. achievement_title is the first candidate for compatibility, and achievement_unlocked is true when at least one candidate is detected. The player name in the lower-left hero panel is not used for this signal. If no checked text is visible in the ROI, the title fields are empty/null. achievement_challenge_disabled is true when the left panel or the settlement card titles line shows the game's 成就挑战已禁用 notice, which means a setting made achievements unobtainable for that run; it is null when the notice is not read, so a missing notice is not proof that achievements were enabled. The platform decides what it means. event is additive evidence for the single active event under the stats block: name, duration_seconds (from a title such as 心之钢（27 秒）), the description lines, every number in them (numbers: text, value, unit) and the joined text. OCRKit reports what it read; the platform compares it with the game version and decides what it means. event is null when no event line is visible. Because the completion mark is an icon that OCR does not read, a line counts as an achievement when a bright check icon follows it (or a text mark is read), so event lines and achievement_titles (at most five) are told apart without a catalog. ai_mark_detected is true when the bottom-right watermark region reads as an AI-generation mark (for example 豆包AI生成) and false otherwise; it is evidence, not a verdict. settlement_card supplies completion, clear time, the card player name (real_player, used when the server-load panel prints none), deaths and skips, map, difficulty, version, run code (labelled 通关代码) and the earned title line (an empty — line gives no titles) when the older banner, stats panel or right panel are not on screen; those older sources win when both are read. mode (the bracketed label under the map), restart_in_seconds, uptime_seconds and server_load come from the right panel; a duration with a missing unit digit is left null instead of guessed. run_code also accepts the unlabeled code printed by the current HUD (strict dddd-dddd-dddd, first digit 1-9) and marks it label:absent.

viewer_player (bottom-left hero card) and real_player are the player identity fields suitable for account matching. The hero card can show a streamer-mode name; newer HUDs also print the player's real in-game name above the server load, which real_player reads from server_load_panel and leaves null when the HUD prints none. OCRKit reports both as evidence and does not choose between them. The central completion banner is not an authoritative player source, so its player name is not included in ChallengeData or field evidence. map_variant is "classic" when the right-panel map label contains 经典版 or 经典, and null when the variant is not detected. run_code is additive, optional OCR evidence from the dedicated settlement run_code_panel / run_code_right_panel ROIs. When both sides contain the same code, OCRKit merges the repeated evidence and reports the ROI(s) that supplied text. When its visible label and all three numeric groups are reliable, OCRKit returns canonical NNNN-NNNN-NNNN; it records the ROI confidence, source, normalization, and one of ok, missing, invalid, ambiguous, or low_confidence. Legacy supported layouts without that field return missing rather than an error. OCRKit never decides whether a run code is eligible for mastery, duplicate, or worth XP. The compatible Bastion release version is intentionally not recorded until the HUD field is actually released.

R2 Object Mode

Configure these environment variables to enable by-object endpoint:

• OCRKIT_R2_ENDPOINT_URL
• OCRKIT_R2_ACCESS_KEY_ID
• OCRKIT_R2_SECRET_ACCESS_KEY
• OCRKIT_R2_REGION_NAME (default: auto)
• OCRKIT_R2_DEFAULT_BUCKET
• OCRKIT_R2_ALLOWED_BUCKETS (comma-separated whitelist)
• OCRKIT_R2_READ_TIMEOUT_SECONDS (default: 10)

For platform evidence, the caller must provide bucket explicitly in the by-object request. Add the platform evidence bucket to OCRKIT_R2_ALLOWED_BUCKETS; do not point OCRKIT_R2_DEFAULT_BUCKET at the platform bucket when it is reserved for OCRKit models or other service-owned objects.

The local OCRKit Studio can use the same read-only R2 credential set to import screenshot sources without a platform API token. Configure the Studio-only bucket and prefix allowlist:

• OCRKIT_STUDIO_R2_BUCKET
• OCRKIT_STUDIO_R2_ALLOWED_PREFIXES (comma-separated, for example uploads/)
• OCRKIT_STUDIO_R2_MAX_OBJECTS (default: 200)
• OCRKIT_STUDIO_R2_MAX_OBJECT_BYTES (default: 26214400)

These settings affect only the local Studio API. Credentials stay server-side; the browser receives object metadata and the selected images are copied into the private ignored Studio batch. Use a dedicated R2 access key with read-only permission scoped to this bucket and prefix. Studio never lists or reads the production model prefix through this import path.

OCR Engine and Model Artifacts

Default engine is rapidocr. Production RapidOCR loads a versioned PP-OCRv6 small artifact manifest from Cloudflare R2; the service image does not contain PaddleOCR or model binaries. It reuses the existing R2 endpoint and credentials, plus:

• OCRKIT_R2_DEFAULT_BUCKET
• OCRKIT_MODEL_MANIFEST_KEY (the initial versioned fallback manifest)
• OCRKIT_MODEL_RELEASE_CHANNEL_KEY (recommended: models/pp-ocrv6-small/channels/stable.json)
• OCRKIT_MODEL_CACHE_DIR (default: /var/lib/ocrkit/models)
• OCRKIT_MODEL_DOWNLOAD_TIMEOUT_SECONDS (default: 30)

When a release channel is configured, it takes precedence over the fallback manifest at container startup. The service verifies the selected immutable artifact before serving traffic. Without either model setting, local development uses RapidOCR's bundled default model.

Rust image preflight

The Rust image core validates the versioned ROI layout manifest, decodes and normalizes training screenshots, exports raw ROI PNGs, records source/crop SHA-256 provenance, maps standard-layout coordinates, and reports aspect-ratio quality warnings. The authoritative layout remains configs/roi_1280x720.yaml; regenerate and check its JSON manifest with:

uv run python scripts/export_layout_manifest.py \
  configs/roi_1280x720.yaml \
  configs/roi_1280x720.manifest.json \
  --check
cargo test --manifest-path rust/Cargo.toml

The native CLI participates in Studio training-data preparation only. Python continues to perform ROI-specific preprocessing, RapidOCR/Vision recognition, human review, and PaddleOCR training. Rust does not replace the production OCR path or make approval decisions. See rust/README.md.

PaddleOCR is only used offline for fine-tuning and export. See training/README.md for the PP-OCRv6 small det/rec label formats, validation, manifest generation, and Cloudflare R2 publication workflow.

uv run python scripts/batch_eval.py

Docker Compose

For a deployed model, copy .env.model.example to a deployment-only .env, fill in the R2 credentials, retain a known-good OCRKIT_MODEL_MANIFEST_KEY for legacy rollback, and set the stable OCRKIT_MODEL_RELEASE_CHANNEL_KEY once after that channel exists. Models use the reserved models/pp-ocrv6-small/ prefix; user screenshots use uploads/. Do not commit that file. The container downloads and verifies the model at startup, then follows the channel without later environment edits.

docker compose up --build -d
docker compose ps

Studio publishes a candidate channel after a fully verified publication; compare and explicitly promote that candidate before the stable channel changes. Restart or recreate the OCRKit container to adopt a newly promoted or rolled-back channel target. The old environment-variable flow remains available for initial migration:

docker compose up -d --force-recreate