Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,11 @@ Format follows [Keep a Changelog](https://keepachangelog.com/); versions are git

### Added
- **D-015 rasm-uniqueness proof (I18/I19)**: bare-typed quotes («قل هو الله احد») are `found` with notice `rasm_completed` and the completed letters when the corpus spells the span one way; competing spellings («إن/أن الله على كل شيء قدير») → `needs_review/rasm_ambiguous` with alternatives + counts. Validator **V7** re-derives the proof. `QuoteResult.rasm` exposes it. 25 new tests.
- **E-057 whole-ayah anchor**: a complete ayah typed with no marker is detected whatever its words' frequency.
- **E-058 phrase-rarity anchor**: short famous texts typed with no marker («إنما الأعمال بالنيات», «لا ضرر ولا ضرار», «إن الله مع الصابرين») are detected; runs extend backwards over leading particles; isnad fragments never anchor. Composer hint: no marker, no hamza needed. 15 tests + 7 smoke cases «as people write».
- **E-061 whole-ayah anchor**: a complete ayah typed with no marker is detected whatever its words' frequency.
- **E-062 phrase-rarity anchor**: short famous texts typed with no marker («إنما الأعمال بالنيات», «لا ضرر ولا ضرار», «إن الله مع الصابرين») are detected; runs extend backwards over leading particles; isnad fragments never anchor. Composer hint: no marker, no hamza needed. 15 tests + 7 smoke cases «as people write».
- Evaluation: category B expectations are now resolved *by construction* from the corpus (`eval/materialize.rasm_expectation`).
- Telegram bot `integrations/telegram` (@BasiraCheckBot): thin client over `/v1/check` and `/v1/check/image`; no model, no storage, texts from `messages/*.json`; optional compose profile `bot`; CI job `bot` (E-059). Replies are Telegram Rich Messages — headings, verbatim pull-quotes, marked differences, tables, collapsible sections, buttons — with a classic-HTML fallback (E-060).
- Bot in production on the VPS (profile `bot`, secrets in `/etc/basira`, shared `BASIRA_EVAL_KEY`, `bot: healthy` deploy log line); `/check?text=` deep link from the bot's «open in Basira» button (E-063). Decision IDs E-057/E-058 of PR #1 renumbered E-061/E-062.

### Verified
pytest 344 · ruff + mypy strict · eval-full **150/150 · unsafe 0 · 0/500** (fixture and full index) · IslamicEval 1B unchanged **78.54 %, false confirmations 2**.
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ Full invariants I1–I17 and validator passes V1–V6: [`SAFETY.md`](SAFETY.md).
| **Verification receipt** — stateless, replayable | `POST /v1/receipt` · `GET /v/{token}?h=` | live |
| **MCP server** — 6 tools for AI assistants | `/mcp` (`BASIRA_MCP=1`) | live |
| Developer gate: sources, grounding rules, measured model catalog | `/v1/sources` · `/v1/rules` · `/v1/models` | live |
| **Telegram bot** — forward a text or image, get the source verbatim | [@BasiraCheckBot](https://t.me/BasiraCheckBot) · `integrations/telegram/` | live |
| **Telegram bot** — forward a text or image, get the source verbatim; «open in Basira» pre-fills the workspace | [@BasiraCheckBot](https://t.me/BasiraCheckBot) · `integrations/telegram/` · runs on the VPS (DEPLOYMENT §3.1) | live |
| Model settings — Genspark key saved once on the server | `/settings` · `PUT /v1/models/config` | live |

## Measured, not claimed
Expand Down Expand Up @@ -132,7 +132,7 @@ eval/ 150 cases, false-alarm generator, English gate eval, IslamicEval
messages/ ar.json / en.json — the ONLY source of user-facing prose
scripts/ bootstrap · smoke · mcp_demo · bench_models · lexicon gate · generators
docs/ ARCHITECTURE · API · SAFETY-adjacent docs · DECISIONS (E-001…E-054) · STATE · adr/ · KNOWLEDGE
.github/ GitHub Actions workflow (ci.yml, active — E-058)
.github/ GitHub Actions workflow (ci.yml, active — E-057)
```

## Documentation map
Expand Down
10 changes: 5 additions & 5 deletions backend/app/extract/anchor.py
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ def _occurs(store: Store, ids: list[int]) -> np.ndarray:
return cand


# E-058 — phrase-rarity acceptance for short unmarked runs. A run of ≥ RARE_MIN_TOKENS tokens that occurs
# E-062 — phrase-rarity acceptance for short unmarked runs. A run of ≥ RARE_MIN_TOKENS tokens that occurs
# verbatim at most RARE_MAX_OCC times in the whole corpus is a quotation, however common its words are
# taken one by one: «إنما الأعمال بالنيات» occurs 7×, «إن الله مع الصابرين» 2×, «لا ضرر ولا ضرار» 9× — while
# formulaic prose is three orders of magnitude more frequent («صلى الله عليه وسلم» 92 083×, «حدثنا عبد الله بن»
Expand Down Expand Up @@ -172,7 +172,7 @@ def detect(store: Store, text: str, *, seed: int = 4, max_cand: int = 4000) -> l
i = 0
min_seed = min(seed, RARE_MIN_TOKENS)
while i + min_seed <= n:
# the seed window is `seed` tokens when available, else the shorter rare-phrase seed (E-058)
# the seed window is `seed` tokens when available, else the shorter rare-phrase seed (E-062)
s = seed if i + seed <= n else min_seed
win = ids[i : i + s]
if any(x < 0 for x in win) or all(toks[i + k].loose in _STOP for k in range(s)):
Expand Down Expand Up @@ -200,7 +200,7 @@ def detect(store: Store, text: str, *, seed: int = 4, max_cand: int = 4000) -> l
# extend BACKWARDS as well: the seed may have started one or more tokens late because the
# leading words were all stop-words («ان الله علي كل …» seeds at «الله»); the quote still
# begins where the corpus agreement begins, and a verdict on a truncated quote is a worse
# verdict (E-058).
# verdict (E-062).
i0 = i
while i0 > 0 and ids[i0 - 1] >= 0 and live.size:
prev = live - 1
Expand All @@ -220,10 +220,10 @@ def detect(store: Store, text: str, *, seed: int = 4, max_cand: int = 4000) -> l
need = 4 if corpus == "tanzil" else 6 # hadith prose is far more formulaic → longer seed
if (
(j - i >= need and len(content) >= 3)
# E-057: a run that IS a complete ayah («قل هو الله أحد») is a quote however common its
# E-061: a run that IS a complete ayah («قل هو الله أحد») is a quote however common its
# words are — the Mushaf's own ayah boundary is the evidence, not word rarity.
or _covers_whole_ayah(store, live, j - i)
# E-058: a short run that is RARE as a phrase («إنما الأعمال بالنيات», «لا ضرر ولا ضرار») and not isnad-shaped
# E-062: a short run that is RARE as a phrase («إنما الأعمال بالنيات», «لا ضرر ولا ضرار») and not isnad-shaped
or (
j - i >= RARE_MIN_TOKENS
and len(content) >= 1
Expand Down
8 changes: 4 additions & 4 deletions backend/tests/test_as_people_write.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
"""«كما يكتب الناس» — unmarked, hamza-less input must be understood (supervisor feedback, 2026-10-05).

A supervisor typed «إنما الأعمال بالنيات» alone and got "zero matches", then added «قال رسول الله …»
and it matched — and warned that judges would conclude «الموقع لا يعمل». E-057/E-058 fix the detector;
and it matched — and warned that judges would conclude «الموقع لا يعمل». E-061/E-062 fix the detector;
D-015 fixes the hamza. These tests pin both, on the fixture (records chosen from it). Phrases that
must NOT be treated as quotations are pinned too (false-alarm side).
"""
Expand All @@ -26,9 +26,9 @@ async def _run(pipeline: Pipeline, text: str) -> CheckResponse:
@pytest.mark.parametrize(
"text",
[
"قل هو الله أحد", # whole ayah, 3 stop-words (E-057)
"قل هو الله أحد", # whole ayah, 3 stop-words (E-061)
"قل هو الله احد", # + unwritten hamza (D-015)
"إن الله مع الصابرين", # ayah fragment, rare phrase (E-058)
"إن الله مع الصابرين", # ayah fragment, rare phrase (E-062)
"ان الله مع الصابرين",
"إنما الأعمال بالنيات", # the supervisor's exact input
"انما الاعمال بالنيات",
Expand Down Expand Up @@ -80,7 +80,7 @@ async def test_prose_and_non_corpus_sayings_are_not_detected(pipeline: Pipeline,


def test_isnad_fragment_does_not_anchor_as_a_rare_phrase(store: Store) -> None:
"""«نافع عن ابن عمر» is rare as a phrase but is a narrator chain, not a matn (E-058 guard)."""
"""«نافع عن ابن عمر» is rare as a phrase but is a narrator chain, not a matn (E-062 guard)."""
spans = detect(store, "حدثنا عبد الله بن يوسف قال أخبرنا مالك عن نافع عن ابن عمر")
assert all(
not (s.n_tokens <= 5 and "نافع" in "x") for s in spans
Expand Down
24 changes: 21 additions & 3 deletions deploy/vps/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ site untouched (Compose replaces the container only after a successful build).
| `docker-compose.prod.yml` | `/opt/basira/docker-compose.prod.yml` | Overlay on `docker-compose.yml`: no public app port, CORS for the site, trusted proxy range, Caddy service |
| `Caddyfile.example` | `/opt/basira/Caddyfile` | Automatic HTTPS, `www` → apex, basic auth on writes to `/v1/models/config` |
| `autodeploy.sh` | `/opt/basira/autodeploy.sh` | Compare server commit with `$BRANCH` of `$REPO`; fetch, rebuild, health-check, log |
| `autodeploy.conf.example` | `/opt/basira/autodeploy.conf` | `REPO` and `BRANCH` to follow |
| `autodeploy.conf.example` | `/opt/basira/autodeploy.conf` | `REPO`, `BRANCH` to follow, `COMPOSE_PROFILES` (`bot` = Telegram bot) |
| `deploy.sh` | `/opt/basira/deploy.sh` | Manual forced redeploy |
| `basira-autodeploy.service` / `.timer` | `/etc/systemd/system/` | Run `autodeploy.sh` every 60 s |
| `basira-deploy.logrotate` | `/etc/logrotate.d/basira-deploy` | Weekly rotation of `/var/log/basira-deploy.log` |
Expand Down Expand Up @@ -44,5 +44,23 @@ curl -s https://basirapp.site/health # build_sha = deployed commi
**Rollback:** `git -C /opt/basira reset --hard <sha> && BUILD_SHA=<sha> docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build`,
and set `BRANCH` to a branch pinned at that commit (or stop the timer) so the next check does not move it forward.

Secrets (admin password, model key) never live in this repository: the hash is only in the server's
`Caddyfile`, and the model key is stored by the app in the `/data` volume.
Secrets (admin password, model key, bot token) never live in this repository: the hash is only in the server's
`Caddyfile`, the model key is stored by the app in the `/data` volume, and the bot token / eval key live in
`/etc/basira/*.env` (600 root).

## Telegram bot (profile `bot`)

```bash
install -d -m 700 /etc/basira
key=$(openssl rand -hex 24)
printf 'BASIRA_EVAL_KEY=%s\n' "$key" > /etc/basira/basira.env
printf 'TELEGRAM_BOT_TOKEN=%s\nBASIRA_EVAL_KEY=%s\n' '<token>' "$key" > /etc/basira/telegram-bot.env
chmod 600 /etc/basira/*.env
sed -i 's/^COMPOSE_PROFILES=.*/COMPOSE_PROFILES=bot/' /opt/basira/autodeploy.conf
/opt/basira/deploy.sh # or wait for the next commit on main
```

`autodeploy.sh` exports `COMPOSE_PROFILES` before `docker compose up`, and after a successful deploy appends
`bot: healthy|unhealthy|missing` (from `docker inspect`) to the log. Without the profile the stack is the site
only — `docker compose up` in a fresh clone is unchanged for judges. Details and the verification list:
`docs/DEPLOYMENT.md` §3.1.
3 changes: 3 additions & 0 deletions deploy/vps/autodeploy.conf.example
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
# Auto-deploy settings. Edit and save — takes effect on the next check (≤ 1 min).
REPO=https://github.com/MoTechSys/Project-Basira.git
BRANCH=main
# Compose profiles to start in addition to the site. "bot" runs the Telegram bot
# (token in /etc/basira/telegram-bot.env, see deploy/vps/README.md). Leave empty for site only.
COMPOSE_PROFILES=bot
8 changes: 8 additions & 0 deletions deploy/vps/autodeploy.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
set -uo pipefail
cd /opt/basira
source /opt/basira/autodeploy.conf
export COMPOSE_PROFILES="${COMPOSE_PROFILES:-}" # e.g. "bot" → also build/run the Telegram bot service
LOG=/var/log/basira-deploy.log
exec 9>/run/basira-deploy.lock
flock -n 9 || exit 0 # a deploy is already running
Expand All @@ -31,6 +32,13 @@ if docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build
done
live=$(curl -s -m 10 "${HEALTH_URL:-https://basirapp.site/health}" | grep -o '"build_sha":"[^"]*"')
echo " ✅ DEPLOYED $BUILD_SHA in $(( $(date +%s) - start ))s · health $code · $live"
if [[ ",$COMPOSE_PROFILES," == *,bot,* ]]; then # bot container health (HEALTHCHECK = polling heartbeat < 90 s)
for _ in $(seq 1 30); do
bot=$(docker inspect --format '{{.State.Health.Status}}' basira-telegram-bot-1 2>/dev/null || echo missing)
[ "$bot" = healthy ] && break; sleep 2
done
echo " bot: ${bot:-missing}"
fi
docker image prune -f >/dev/null
else
echo " ❌ BUILD FAILED — site still runs the previous version. Details: /root/build.log"
Expand Down
17 changes: 17 additions & 0 deletions deploy/vps/docker-compose.prod.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,26 @@ services:
basira:
ports: !reset []
expose: ["8000"]
env_file:
- path: /etc/basira/basira.env # BASIRA_EVAL_KEY (600 root) — lets the bot bypass the per-IP limit
required: false
environment:
BASIRA_CORS_ORIGINS: https://basirapp.site,https://www.basirapp.site
BASIRA_TRUSTED_PROXIES: 172.16.0.0/12
FORWARDED_ALLOW_IPS: 172.16.0.0/12
BUILD_SHA: ${BUILD_SHA:-prod}
mem_limit: 2g
logging:
driver: json-file
options: { max-size: "10m", max-file: "3" }
telegram-bot: # started only with COMPOSE_PROFILES=bot (autodeploy.conf)
env_file: !override [/etc/basira/telegram-bot.env] # TELEGRAM_BOT_TOKEN + BASIRA_EVAL_KEY (600 root); replaces ./integrations/telegram/.env
environment:
BASIRA_API_URL: http://basira:8000
BASIRA_WEB_URL: https://basirapp.site
logging:
driver: json-file
options: { max-size: "10m", max-file: "3" }
caddy:
image: caddy:2-alpine
restart: unless-stopped
Expand All @@ -17,6 +31,9 @@ services:
- caddy-data:/data
- caddy-config:/config
depends_on: [basira]
logging:
driver: json-file
options: { max-size: "10m", max-file: "3" }
volumes:
caddy-data:
caddy-config:
Loading
Loading