Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,15 @@ jobs:
- run: npm run lint && npm run typecheck && npm test -- --run
- run: npm run build
- run: npm audit --omit=dev --audit-level=high
bot:
runs-on: ubuntu-latest
defaults: { run: { working-directory: integrations/telegram } }
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: { python-version: "3.12", cache: pip }
- run: pip install -e '.[dev]' && pip install -e '../../backend'
- run: ruff check . && ruff format --check . && mypy --strict . && pytest -q
secrets:
runs-on: ubuntu-latest
steps:
Expand Down
1 change: 1 addition & 0 deletions AI_USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@

### المزوّدون المدعومون
أي واجهة متوافقة مع OpenAI: Gemini (مجاني 1,500/يوم)، Groq، OpenAI. يُضبط بـ `LLM_BASE_URL/LLM_API_KEY/LLM_MODEL`. التقييم الرسمي (`eval/REPORT.md`) يُشغَّل بـ `mock` ليكون **قابلًا للتكرار بالبايت**؛ ويُعاد بمزوّد حقيقي مع سطر «تباين الاستخراج».
- بوت تيليجرام (`integrations/telegram`) لا يحوي أي نموذج؛ هو عميل رقيق لواجهة بصيرة البرمجية.

## 2) في البناء (أثناء التطوير)

Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Format follows [Keep a Changelog](https://keepachangelog.com/); versions are git
- **E-057 whole-ayah anchor**: a complete ayah typed with no marker is detected whatever its words' frequency.
- **E-058 phrase-rarity anchor**: short famous texts typed with no marker («إنما الأعمال بالنيات», «لا ضرر ولا ضرار», «إن الله مع الصابرين») are detected; runs extend backwards over leading particles; isnad fragments never anchor. Composer hint: no marker, no hamza needed. 15 tests + 7 smoke cases «as people write».
- Evaluation: category B expectations are now resolved *by construction* from the corpus (`eval/materialize.rasm_expectation`).
- Telegram bot `integrations/telegram` (@BasiraCheckBot): thin client over `/v1/check` and `/v1/check/image`; no model, no storage, texts from `messages/*.json`; optional compose profile `bot`; CI job `bot` (E-059). Replies are Telegram Rich Messages — headings, verbatim pull-quotes, marked differences, tables, collapsible sections, buttons — with a classic-HTML fallback (E-060).

### Verified
pytest 344 · ruff + mypy strict · eval-full **150/150 · unsafe 0 · 0/500** (fixture and full index) · IslamicEval 1B unchanged **78.54 %, false confirmations 2**.
Expand Down
8 changes: 7 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ URL ?= http://localhost:8000
.PHONY: help bootstrap fetch fetch-translations index fixture \
lint fmt test gates smoke web-install web-lint web-test web-build web-gates \
serve serve-mcp mcp-demo eval eval-full eval-english eval-english-picker \
islamiceval islamiceval-1a scholar docker clean
islamiceval islamiceval-1a scholar docker clean bot-install bot-test

help: ## list targets
@awk 'BEGIN{FS=":.*## "} /^[a-z0-9-]+:.*## /{printf " \033[1m%-20s\033[0m %s\n",$$1,$$2}' $(MAKEFILE_LIST)
Expand Down Expand Up @@ -78,3 +78,9 @@ scholar: ## scholar-lens probe against a running server (URL=...)
clean: ## remove caches and build output (keeps corpora and venv)
find . -type d \( -name __pycache__ -o -name .pytest_cache -o -name .mypy_cache -o -name .ruff_cache \) -prune -exec rm -rf {} +
rm -rf frontend/dist eval/results

# ---- Telegram bot (integrations/telegram) ---------------------------------------------------------
bot-install: ## venv + deps for the Telegram bot
cd integrations/telegram && python3 -m venv .venv && .venv/bin/pip install -q -e '.[dev]'
bot-test: ## bot gates: ruff + mypy strict + pytest
cd integrations/telegram && .venv/bin/ruff check . && .venv/bin/ruff format --check . && .venv/bin/mypy --strict . && .venv/bin/pytest -q
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ Full invariants I1–I17 and validator passes V1–V6: [`SAFETY.md`](SAFETY.md).
| **Verification receipt** — stateless, replayable | `POST /v1/receipt` · `GET /v/{token}?h=` | live |
| **MCP server** — 6 tools for AI assistants | `/mcp` (`BASIRA_MCP=1`) | live |
| Developer gate: sources, grounding rules, measured model catalog | `/v1/sources` · `/v1/rules` · `/v1/models` | live |
| **Telegram bot** — forward a text or image, get the source verbatim | [@BasiraCheckBot](https://t.me/BasiraCheckBot) · `integrations/telegram/` | live |
| Model settings — Genspark key saved once on the server | `/settings` · `PUT /v1/models/config` | live |

## Measured, not claimed
Expand Down
12 changes: 12 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,5 +29,17 @@ services:
timeout: 5s
retries: 3

telegram-bot: # optional: docker compose --profile bot up
profiles: ["bot"]
build: ./integrations/telegram
env_file: [./integrations/telegram/.env]
environment:
BASIRA_API_URL: http://basira:8000
depends_on: [basira]
read_only: true
tmpfs: [/tmp]
mem_limit: 128m
restart: unless-stopped

volumes:
basira-data:
2 changes: 2 additions & 0 deletions docs/DECISIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,3 +89,5 @@
| E-058 | 2026-10-06 | **Phrase-rarity anchor + backward extension + isnad guard.** `extract/anchor.detect` accepts an unmarked run of ≥ 3 tokens that occurs ≤ 60× verbatim in the whole corpus (with ≥ 1 content word, not isnad-shaped), and extends every run backwards over leading stop-words. Composer shows «لا يلزم قال تعالى… ولا الهمزات». | Supervisor feedback (2026-10-05): «إنما الأعمال بالنيات» alone → zero matches, with «قال رسول الله» → found; «judges will say the site does not work». Phrase rarity separates quotations (7×, 2×, 9×) from formulaic prose (92 083×, 3 248×, 1 191×) by three orders of magnitude; the band 27 ↔ 105 is empty. The old 6-content-word rule hid every short famous hadith. Verified: 15 new tests (`test_as_people_write.py`), smoke +7 unmarked cases, eval-full 150/150 · 0/500 unchanged. | `backend/app/extract/anchor.py`, `scripts/smoke.py`, `frontend/src/{Check.tsx,i18n.ts}` |
| E-057 | 2026-10-05 | **CI activated; non-product material removed from the tree.** The workflow lives in `.github/workflows/ci.yml` and builds the sha256-pinned corpora → index → fixture before tests, so corpus-backed tests run instead of skipping. | Measured on a clean clone: pytest 304 passed, eval 150/150, unsafe 0. | `.github/workflows/ci.yml` |
| E-058 | 2026-10-05 | **Release 0.3.1 hygiene.** Runtime deps and Python 3.12 declared; `run_1b.py` aligned with `Pipeline.check()` and IslamicEval re-measured (1B 78.54 %, false confirmations 2); docs point only at public documents; BUILD_SHA passed into the image. | A clean install failed at collection; `make islamiceval` crashed with ValueError. | `backend/pyproject.toml`, `eval/islamiceval/`, `Dockerfile`, `docs/` |
| E-059 | 2026-10-06 | **Telegram bot as a thin API client** (`integrations/telegram`). Long polling by default: no public port, no TLS endpoint, no webhook secret to manage, works behind NAT; webhook is opt-in. No storage: text and image bytes exist only for one request (ADR-004); logs carry counts and latency only. Every sentence about a quotation is a `messages/*.json` template; bold ranges come from the API `diff`. | 73 tests (render/client/handlers through the real PTB `Application`); live 11/11 private scenarios + channel forward + group `/check`. Privacy mode drops plain @mentions (0 updates measured) → groups use `/check`. | `integrations/telegram/**`, `Makefile`, `.github/workflows/ci.yml`, `docker-compose.yml` |
| E-060 | 2026-10-06 | **Bot replies as Telegram Rich Messages (Bot API 10.3).** One RTL rich message per result: heading + status strip, the user's quote as a cited blockquote, `source_text` as a pull-quote credited with `ref_label_ar` (one per ayah), `<mark>` from the API `diff`, a compact metadata table, collapsible notices/positions/full record, in-message url/copy buttons. Classic HTML (4 096-char split) is the automatic fallback on any rejected rich call. | Live: 16/16 scenarios on the rich path, 0 rejected; 105 tests incl. Rich-HTML validation and fallback. Previous classic layout read as an undifferentiated wall of text (owner feedback). | `integrations/telegram/bot/rich.py`, `handlers.py`, `docs/design/evidence/telegram/` |
4 changes: 4 additions & 0 deletions docs/INTEGRATIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,3 +120,7 @@ is what makes two faithful encodings of the Mushaf agree. `remote returned 1036
* OAuth / user accounts (nothing to protect; contradicts «لا نحفظ شيئًا»).
* Semantic/embedding search as a verification primitive (non-deterministic; I7 reproducibility).
* Serving tafsir, translations or any interpretive text (red line: never generate or curate religious text beyond the pinned corpora).

## 4. Telegram bot

[@BasiraCheckBot](https://t.me/BasiraCheckBot) — `integrations/telegram/` (README there). Forward any text or image; the bot calls `POST /v1/check` or `POST /v1/check/image` and renders the response with `messages/ar.json` only: status, verbatim `source_text`, reference, bold differences from `diff`, links, transparency line. No model in the bot, nothing stored. Run: `docker compose --profile bot up`. Decision: E-059.
Binary file added docs/design/evidence/telegram/01_start.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/design/evidence/telegram/02_found_quran.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/design/evidence/telegram/04_not_found.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/design/evidence/telegram/09_sources.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
17 changes: 17 additions & 0 deletions docs/design/evidence/telegram/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Telegram bot — evidence (2026-10-06)

Rendered from the **rich_message blocks Telegram stored** for real replies of @BasiraCheckBot (fetched back with a
user session after `scripts/live_e2e.py` ran against https://basirapp.site). The drawing is a Telegram-like dark
theme (`integrations/telegram/scripts/preview_blocks.py`); the structure — headings, pull-quotes, tables, collapsible sections, buttons and
their styles — is exactly what the server returned.

| File | Input |
|---|---|
| `01_start.png` | `/start` |
| `02_found_quran.png` | «قال تعالى: ﴿إن الله مع الصابرين﴾» |
| `03_needs_review_quran.png` | «قال تعالى: ﴿إن الله علي كل شيء قدير﴾» — «علي» / «عَلَىٰ» marked |
| `04_not_found.png` | «قال ﷺ: «الدين المعاملة»» — search buttons only |
| `05_partial_hadith.png` | «طلب العلم فريضة على كل مسلم ومسلمة» — «ومسلمة» marked |
| `07_image_hadith.png` | an image of a hadith → OCR text first, `needs_review/image_unconfirmed` |
| `08_channel_forward.png` | a channel post forwarded to the bot (ayah + hadith, «رواه البخاري») |
| `09_sources.png` | `/sources` |
12 changes: 12 additions & 0 deletions integrations/telegram/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Copy to .env (git-ignored). Never commit a real token.
TELEGRAM_BOT_TOKEN= # from @BotFather
BASIRA_API_URL=https://basirapp.site # or http://basira:8000 inside docker compose
BASIRA_WEB_URL=https://basirapp.site
BASIRA_EVAL_KEY= # optional, sent as X-Eval-Key
BOT_RATE_PER_MIN=10 # per user, in memory
MAX_TEXT=5000
BOT_RICH=1 # Telegram Rich Messages (Bot API 10.3); 0 = classic HTML
# Webhook mode (optional; polling when empty)
TELEGRAM_WEBHOOK_URL=
TELEGRAM_WEBHOOK_SECRET=
PORT=8080
10 changes: 10 additions & 0 deletions integrations/telegram/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
FROM python:3.12-slim
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 BOT_HEARTBEAT_FILE=/tmp/basira-bot.heartbeat
WORKDIR /app
COPY pyproject.toml ./
COPY bot/ bot/
RUN pip install --no-cache-dir . && useradd -r -u 10001 bot
USER bot
HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
CMD python -c "import time,sys;sys.exit(0 if time.time()-int(open('/tmp/basira-bot.heartbeat').read())<90 else 1)"
CMD ["python", "-m", "bot.main"]
73 changes: 73 additions & 0 deletions integrations/telegram/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# Basira Telegram bot

A thin client over the public Basira API. Send or forward a text or an image that quotes the Quran or Hadith; the
bot replies with the status, the source text **verbatim**, its reference, the differing letters **marked** (from the
API `diff`), source/search buttons and the transparency line. Live: **[@BasiraCheckBot](https://t.me/BasiraCheckBot)**.

## Layout — Telegram Rich Messages (Bot API 10.3)
Each result is **one** right-to-left rich message (`sendRichMessage`, then `editMessageText.rich_message` over a
«جارٍ الفحص…» placeholder), up to 32 768 characters instead of 4 096:

| Block | Content (all from the API or `messages/ar.json`) |
|---|---|
| `<h2>` + status strip | 🔎 نتيجة الفحص · ✅ 1 وُجد · ⚠️ 1 يحتاج مراجعة · اقتباسان |
| `<h3>` per quote | state icon + label + kind, `1/2` counter |
| paragraph | `messages.status[message_key]` |
| `<blockquote>` + cite «نصك — كما كتبته» | the user's quote, differing words `<mark>`ed |
| `<aside>` pull-quote + credit = `ref_label_ar` | `source_text` verbatim (one per ayah for multi-ayah quotes), differences `<mark>`ed |
| `<details>` 📜 | the full record when only the matched window is shown |
| compact bordered `<table>` | source · collection · positions · diff kind · review reason · claimed reference |
| `<details>` ℹ️ / 📍 | notices; other positions as links |
| `<tg-button-row>` | 📖 المصدر (primary) · search links · 📋 copy source text (≤ 256 chars, never for images) |
| `<footer>` | transparency notice + determinism fingerprint, «افتح موقع بصيرة» (success) |

If a rich call is rejected (older Bot API, unexpected markup) the same content is sent as classic HTML split under
4 096 characters — a reply is never lost (`BOT_RICH=0` forces classic). Evidence: `docs/design/evidence/telegram/`.

## What the bot does not do
- It holds no model and writes no religious text. Every sentence about a quotation is a template from
`messages/ar.json` (served by `GET /v1/messages/ar`); `source_text` / `ref_label_ar` are inserted as received.
- It issues no judgement; `tests/test_strings.py` and `tests/test_render.py` run the backend lexicon scanner.
- It stores nothing (ADR-004): no database, no files; image bytes live in a `BytesIO` closed after the request;
logs carry chat type, input kind, status counts and latency only (`test_logs_never_contain_user_text_or_ids`).
- An image is never `found` (the API enforces it; the bot shows the OCR text first, under «راجعه»).

## Create a bot (@BotFather)
1. `/newbot` → name → username ending in `bot`.
2. Copy the token into `.env` as `TELEGRAM_BOT_TOKEN` (never into git).
3. `/setprivacy` → Enable (the bot then sees only commands and replies in groups).

## Run
```bash
make bot-install && make bot-test # from the repo root
cd integrations/telegram && cp .env.example .env # fill the token
set -a; . ./.env; set +a; .venv/bin/python -m bot.main # long polling
docker compose --profile bot up -d # with the API container (BASIRA_API_URL=http://basira:8000)
```
Webhook mode: set `TELEGRAM_WEBHOOK_URL` and `TELEGRAM_WEBHOOK_SECRET` (16–256 chars); the bot listens on `PORT`
at `/telegram`.

## Usage and limits
| Input | Behaviour |
|---|---|
| `/start`, `/help` | welcome, transparency and privacy lines from `messages/ar.json`, site button |
| text / forward | `POST /v1/check` → one rich message per result (classic fallback split under 4096 chars) |
| photo or PNG/JPEG/WebP document ≤ 6 MB | `POST /v1/check/image` (field `image`) → OCR text, then results |
| `/limits`, `/sources` | `GET /v1/rules?ui_lang=ar` (first 1500 chars), `GET /v1/sources` |
| groups | `/check <text>` or `/check` as a reply to a text/photo; plain messages are ignored |
| > 5000 chars | refused locally with `errors.text_too_long` |
| API 413/422/429/503 | `error.message_ar` verbatim (429 adds «حاول بعد دقيقة») |

Rate limit 10 messages/minute/user (in memory). Timeouts: 30 s text, 60 s image; a «جارٍ الفحص…» placeholder is
edited into the result. **Known limit:** with privacy mode on, Telegram does not deliver plain `@mention` messages
to bots (measured: 0 updates), so in groups use `/check`; mentions work only if privacy is disabled or the bot is
an admin.

## Measured (2026-10-06)
- `ruff check . && ruff format --check . && mypy --strict . && pytest -q` → **105 passed**, 0 lint/type errors
(rich documents validated against the Rich-HTML tag rules, lexicon-scanned, size-checked; classic fallback tested).
- Live, real account → @BasiraCheckBot → https://basirapp.site, rich path (`scripts/live_e2e.py`): **11/11** private
scenarios (start, found, needs_review with «علي»/«عَلَىٰ» marked, not_found + search buttons, partial_match,
refusal, no quotes, sources, limits, unsupported file, image → `needs_review/image_unconfirmed`) and **5/5** channel /
group scenarios (forward from a channel, plain text ignored, `/check` reply, `/check <text>`, `/check` usage);
0 rich calls rejected by Telegram.
7 changes: 7 additions & 0 deletions integrations/telegram/bot/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
"""Basira Telegram bot — a thin client over the public Basira API (POST /v1/check, /v1/check/image).

The bot holds no model, writes no religious text and stores nothing: every word shown about a quotation comes
from the API response or from ``messages/*.json`` (served at ``GET /v1/messages/{lang}``).
"""

__version__ = "0.1.0"
Loading
Loading