Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,16 @@ Read `CLAUDE.md` and the deployment runbook before changing runtime behavior.
- Verify version/review requirements and the selected worktree before `gh ship`. Distinguish implemented, reviewed, merged, deployed and live-tested in reports.
- Currency/FX functionality belongs in ExpenseSyncBot. This bot retains a general calculator and date/time arithmetic, not exchange-rate services.

## Analyzing a problem: read the production logs first

Every analysis of a reported problem (a wrong or silent bot answer, an error, a stuck job) starts from the production logs, before any code reading, hypothesis or fix. The user's account of what the bot did is the symptom; the logs are the evidence of which tool calls ran, what each returned and what was delivered. Commands are in `docs/reference/deploy-runbook.md` (Diagnostics); use these three sources and cite what you found:

1. **Per-chat AI debug logs** (`AI_DEBUG_LOGS=true`): `logs/chats/<chatId>/<UTC timestamp>.log` in the container holds the system prompt, history, every round's tool calls and results, the model text and the FINAL section. Never `cat` a whole log into the conversation (it is calendar data, Telegram ids and a long prompt). Copy the newest file to a private local file outside the repository (`umask 077; f=$(mktemp /tmp/hcb-chat.XXXXXX)`: an absolute template, because a bare `mktemp` honours `TMPDIR`, which may point inside the checkout), read only what you need with `grep -n` and line ranges (the relevant rounds and the FINAL section), and delete the copy when done. Compare the model's last text with FINAL: they differ when a guard replaced the answer.
2. **Docker logs** (pino JSON), around the incident time: levels 40/50/60 (a tool result that says `ERROR` in the chat log usually has its stack here, e.g. `BUTTON_DATA_INVALID` from `sendButtons`) **and** the records that confirm a delivery. FINAL, or a guard-replaced answer, is the reply the bot intended to deliver, recorded before Telegram delivery and cut to 600 characters: it is not proof of delivery. Whether it was delivered is read only from records that exist: `AI request metric` (level 30, one per request) carries `deliveryOutcome` (`delivered`, `fallback` or `discarded`), where `delivered` means the final write did not throw, so a failed final edit or send beside it is logged at level 50 as `Finalize edit failed` or `Finalize send failed` (a throw as `Writer finalize failed`); an invitation shows `Delivered via bot API` or `Delivered via the inviter's own Telegram account` (level 30), `Bot API + inviter session failed — deep-link fallback attempted` (level 30: not delivered to the invitee) or `Delivery chain failed` (level 50). A successful `ask_user` question send logs nothing, so a **missing delivery record is unknown, not failure**: say it is unconfirmed, and do not turn it into a finding or a replay without other evidence (an error record, the user's account of what they saw).
3. **Host logs and evidence**: `/opt/hypercal/logs/` on the odroid (`failed-<sha>-<stamp>/`, `release-*`, `healthcheck*`, `backup*`); `/opt/hypercal/stage/logs` and `/opt/hypercal/stage/evidence` for the stage bot.

Match the deployed revision (`docker ps` image tag) to the code you read. Raw logs stay out of the repository, issues, pull requests and commit messages; quote short anonymized excerpts only (no names, titles, ids or chat text). The tool result is the truth, not the model's explanation of it: a model text that blames "the system" must be checked against the actual result before it becomes a finding.

## Heavy work runs on the server

This workstation is shared by many agent sessions, the review queue and the owner's own
Expand Down
49 changes: 49 additions & 0 deletions docs/plans/2026-10-31-secretary-invitation-incident.md

Large diffs are not rendered by default.

21 changes: 18 additions & 3 deletions docs/reference/deploy-runbook.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,24 @@ curl https://hypercal.mextner.com/ready
system prompt, history, tool calls, responses. Enabled via `AI_DEBUG_LOGS=true`.

```bash
# Latest log for a chat (logs inside container at /app/logs/):
ssh root@odroidn2 'docker exec hypercal-bot ls -lt /app/logs/chats/5153477378/ | head -3'
ssh root@odroidn2 'docker exec hypercal-bot cat /app/logs/chats/5153477378/<timestamp>.log'
# Latest log for a chat (logs inside container at /app/logs/). Copy it to a private temp file and
# read ranges; never cat a whole chat log into a session (calendar data, Telegram ids):
ssh root@odroidn2 'docker exec hypercal-bot ls -lt /app/logs/chats/<chatId>/ | head -3'
# Absolute template: a bare `mktemp` honours TMPDIR, which can point inside the checkout.
umask 077; f=$(mktemp /tmp/hcb-chat.XXXXXX) # mode 0600, outside the repository
ssh root@odroidn2 'docker exec hypercal-bot cat /app/logs/chats/<chatId>/<timestamp>.log' > "$f"
grep -n '^## ROUND\|^## FINAL\|^## END' "$f"; sed -n '<from>,<to>p' "$f"
rm -f "$f"

# Docker logs around an incident: warnings and errors plus the records that exist for a delivery.
# A FINAL answer is recorded before delivery and does not prove it, and a successful ask_user send
# logs nothing: a missing delivery record is unknown, not failure.
# AI request metric (level 30; deliveryOutcome delivered|fallback|discarded)
# Finalize edit failed / Finalize send failed / Writer finalize failed (level 50)
# Delivered via bot API / Delivered via the inviter's own Telegram account (level 30, invitation sent)
# Bot API + inviter session failed — deep-link fallback attempted (level 30, invitation not delivered)
# Delivery chain failed (level 50)
ssh root@odroidn2 'docker logs --since 3h hypercal-bot 2>&1 | grep -E "\"level\":(40|50|60)|AI request metric|Delivered via|deep-link fallback attempted"'
```

## Shared Server
Expand Down
1 change: 1 addition & 0 deletions docs/reference/telegram-limits.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ When text may exceed 4 096 chars, use `splitMessage()` from `src/utils/telegram.
- Max **8 buttons per row**, **100 buttons total**
- Total `reply_markup` JSON: **10 KB**
- `callback_data` per button: **64 bytes** (UTF-8). Exceeding -> `400 BUTTON_DATA_INVALID`.
- ask_user buttons (`src/services/ai/ask-user-buttons.ts`): the answer rides in `ai_btn:<text>` while that fits in 64 bytes; a longer option is sent as `ai_opt:<index>` and the tap handler reads the answer from the tapped button's label in the message's own keyboard. A 2026-10-31 incident (`BUTTON_DATA_INVALID` for a 45-character Russian option) was the byte-vs-character trap. `handleAskUser` rejects only an empty question or option and more than 100 buttons (the working limit above, which carries no source link); it enforces no label-length or keyboard-size limit, because none is documented. Whether Telegram carries a very long button label whole is unverified (#914). A tap is acknowledged by editing `question + "\n\n✅ " + answer` into the message; past the 4096-character message limit that edit fails, so the tap handler falls back to the answer alone, shortens an answer over 4000 characters, writes it on one line (the "already answered" test looks for a final `✅ ...` line), and also remembers the answered message in process memory, so a failed edit cannot let a second tap reach the AI.

## Commands
- Command name: **1-32 chars** (lowercase a-z, 0-9, `_`)
Expand Down
4 changes: 4 additions & 0 deletions docs/specs/06-sharing-social.md
Original file line number Diff line number Diff line change
Expand Up @@ -705,12 +705,14 @@ share_event(params: {
send_invitation(params: {
event_id: number;
invitee_id: number; // telegram_user_id
owner_id?: number; // secretary with write access: the calendar owner whose event it is
}): Promise<{ success: boolean; invitation_id?: number; deep_link?: string }>

// Check invitation status
get_invitation_status(params: {
invitation_id?: number;
event_id?: number; // get all invitations for event
owner_id?: number; // secretary (read or write access) reading the owner's event
}): Promise<Invitation[]>

// Update sharing settings
Expand All @@ -727,6 +729,8 @@ set_event_visibility(params: {
}): Promise<{ success: boolean }>
```

**Secretary invitations.** `send_invitation`, `resend_invitation` and `cancel_invitation` accept `owner_id` and need active `write` secretary access to that owner; `get_invitation_status` needs read access. The invitation's `inviter_id` is the event owner, not the secretary: cards, RSVP notices, the roster and reschedule all read the event as `(event_id, inviter_id)`. Delivery is Bot API first; on failure only the deep link goes to the secretary's private chat. No connected Telegram (MTProto) session sends for the owner. The requester can be invited with their own Telegram ID; inviting the owner is refused (the owner is the organizer). The user picker (`pick_users`) does not yet apply to a secretary-managed event.

---

## 12. BullMQ Jobs
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "hypercalendarbot"
version = "0.3.1"
version = "0.3.2"
requires-python = ">=3.12"
dependencies = [
"pyrofork>=2.3",
Expand Down
204 changes: 169 additions & 35 deletions src/bot/handlers/callback/ai-routes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,12 @@

import type { Lang } from '../../../config/constants.ts';
import { CB, t, toLang } from '../../../config/constants.ts';
import {
AI_BUTTON_ACTION,
AI_OPTION_ACTION,
type InlineKeyboardRows,
parseAiOptionPayload,
} from '../../../services/ai/ask-user-buttons.ts';
import {
approveDeletes,
chooseTargets,
Expand All @@ -17,7 +23,9 @@ import { cmdLogger } from '../../../utils/logger.ts';
import { isGroup } from '../../group-context.ts';
import type { BotCallbackContext } from '../../types.ts';
import { continueWithAgent } from '../agent-continuation.ts';
import type { CallbackDispatch, CallbackRouteDeps } from './types.ts';
import type { CallbackDispatch, CallbackHandlerOpts, CallbackRouteDeps } from './types.ts';

type CallbackHandler = NonNullable<ReturnType<CallbackDispatch['get']>>;

/**
* Parse ai_btn payload into answer text and optional group restriction.
Expand All @@ -41,15 +49,100 @@ export function parseAiBtnPayload(
return { answerText: payload };
}

/** What a tap on an ask_user button means: the answer, and the one user allowed to give it in a group. */
interface AiButtonTap {
/** Undefined for an `ai_opt` tap whose message no longer carries the tapped button: answered already, or inaccessible to the bot. */
answerText: string | undefined;
restrictedToUserId?: number;
}

/**
* Reads a tap on either ask_user button form (`ai_btn`, `ai_opt`). `keyboard` is the inline keyboard
* of the tapped message, where an `ai_opt` button's own label is the answer. Undefined when `data`
* is not an ask_user button or its payload is malformed.
*/
export function readAiButtonTap(
data: string,
inGroup: boolean,
keyboard: InlineKeyboardRows | undefined,
): AiButtonTap | undefined {
const firstColon = data.indexOf(':');
const action = firstColon === -1 ? data : data.slice(0, firstColon);
const payload = firstColon === -1 ? '' : data.slice(firstColon + 1);
if (action === AI_BUTTON_ACTION) return parseAiBtnPayload(payload, inGroup);
if (action !== AI_OPTION_ACTION) return undefined;
const option = parseAiOptionPayload(payload);
if (!option) return undefined;
const tapped = keyboard?.flat().find((button) => button.callback_data === data);
return { answerText: tapped?.text, ...option };
}

interface DeleteReportLabels {
deleteDone: (count: number) => string;
deleteKeptPast: (count: number) => string;
deleteFailed: (titles: string) => string;
}

/** Telegram refuses a message over 4096 characters; the "✅ " marker and an ellipsis ride on top of the answer. */
const MAX_ACKNOWLEDGED_ANSWER_CHARS = 4000;

/**
* The answer as shown after "✅ ". One line, because the "already answered" test needs the marker
* line to end the text, and short enough to be an acceptable message on its own whatever the label.
*/
function acknowledgedAnswer(answerText: string): string {
const oneLine = answerText.replace(/\s*[\r\n]+\s*/g, ' ');
return oneLine.length > MAX_ACKNOWLEDGED_ANSWER_CHARS
? `${oneLine.slice(0, MAX_ACKNOWLEDGED_ANSWER_CHARS)}…`
: oneLine;
}

/** A question whose text already ends in a "✅ answer" line: the edit that records an answer landed. */
const ANSWERED_QUESTION = /(?:^|\n\n)✅ [^\n]*$/;

const ANSWERED_MEMORY_LIMIT = 1000;

/**
* Remembers which question messages were answered, for the taps a client still sends from a message
* it shows unanswered (the acknowledgement edit failed, or two taps raced before either edit landed).
* Bounded to the newest ANSWERED_MEMORY_LIMIT messages. A restart forgets it, as it forgets the
* pending AI turn it guards.
*/
function createAnsweredMessages() {
const answered = new Set<string>();
const keyOf = (ctx: BotCallbackContext) =>
ctx.message?.id === undefined ? undefined : `${ctx.chatId}:${ctx.message.id}`;
return {
has: (ctx: BotCallbackContext) => {
const key = keyOf(ctx);
return key !== undefined && answered.has(key);
},
add: (ctx: BotCallbackContext) => {
const key = keyOf(ctx);
if (key === undefined) return;
answered.add(key);
// Oldest first: a Set iterates in insertion order.
const oldest = answered.values().next();
if (answered.size > ANSWERED_MEMORY_LIMIT && !oldest.done) answered.delete(oldest.value);
},
release: (ctx: BotCallbackContext) => {
const key = keyOf(ctx);
if (key !== undefined) answered.delete(key);
},
};
}

/**
* One memory for the whole process, not one per handler: src/bot/index.ts builds the callback handler
* (and so these routes) afresh for every update, so state held inside the handler is empty on each tap.
* The bot runs as a single container (docker-compose.yml), so process memory sees every tap; Redis
* would only add a round trip and a failure mode to a guard that the pending AI turn does not outlive.
*/
const answeredMessages = createAnsweredMessages();

async function recordAskUserAnswer(ctx: BotCallbackContext, question: string, answerText: string): Promise<boolean> {
const entities = ctx.message?.entities?.map((entity) => entity.payload) ?? [];
const answered = `✅ ${answerText}`;
const answered = `✅ ${acknowledgedAnswer(answerText)}`;
const kept = question ? `${question}\n\n${answered}` : answered;
try {
try {
Expand Down Expand Up @@ -92,6 +185,77 @@ async function reportDeleteOutcome(
}
}

/**
* Acknowledges the tap, records the answer in the chat and starts the AI continuation. A callback
* query too old to answer says nothing about the user's answer, so that failure is only logged.
*/
async function acknowledgeAndContinue(
ctx: BotCallbackContext,
userId: number,
question: string,
answerText: string,
onAiButtonClick: CallbackHandlerOpts['onAiButtonClick'],
): Promise<void> {
await ctx.answer().catch((err: unknown) => cmdLogger.warn({ err }, 'Failed to answer the ask_user tap'));
const recorded = await recordAskUserAnswer(ctx, question, answerText);
if (!recorded) return;
const cbChatId = ctx.chatId;
if (onAiButtonClick && cbChatId) {
onAiButtonClick(userId, cbChatId, answerText).catch((e) => {
cmdLogger.error({ err: e }, 'AI button continuation failed');
});
}
}

/** Handler of both ask_user button forms: records the answer once and hands it to the AI. */
function createAskUserTapHandler(onAiButtonClick: CallbackHandlerOpts['onAiButtonClick']): CallbackHandler {
return async (ctx, _payload, _parts, user, data) => {
const tap = readAiButtonTap(data, isGroup(ctx), ctx.message?.replyMarkup?.payload.inline_keyboard);
if (!tap) {
cmdLogger.warn({ userId: user.telegram_id, action: data.split(':')[0] }, 'ask_user button data is malformed');
await ctx.answer();
return;
}
const { answerText, restrictedToUserId } = tap;
const lang = (user.language ?? 'en') as Lang;

// In groups, only the user who triggered the question can answer
if (restrictedToUserId !== undefined && ctx.from.id !== restrictedToUserId) {
await ctx.answer({ text: t(lang).callbackErrors.notYourQuestion, show_alert: false });
return;
}

// Keep the question: after a destructive confirmation it is the user's only record of
// what they agreed to. Appending at the end leaves the entity offsets valid.
const question = ctx.message?.text ?? '';
// A later tap from a client that still shows the buttons finds an answer already recorded,
// the same option or another one: the AI must not get a second answer (a destructive
// confirmation would run again, or run after the user declined it). The recorded answer is
// either the "✅" line of the edited message or, when that edit failed or has not landed yet,
// this process's memory of the message. Answering removes the keyboard, so a late `ai_opt`
// tap has no label to read: this test comes before that one.
if (ANSWERED_QUESTION.test(question) || answeredMessages.has(ctx)) {
await ctx.answer();
cmdLogger.info({ userId: user.telegram_id }, 'ai_btn tap on an already answered question ignored');
return;
}
if (answerText === undefined) {
cmdLogger.warn({ userId: user.telegram_id }, 'ai_opt tap found no matching button in the tapped message');
await ctx.answer({ text: t(lang).callbackErrors.unavailable, show_alert: true });
return;
}
// Claimed before the first await: a second tap arriving while the edit is in flight sees it.
answeredMessages.add(ctx);
try {
await acknowledgeAndContinue(ctx, user.telegram_id, question, answerText, onAiButtonClick);
} catch (err) {
// The answer never reached the AI: release the claim so the user's retap is processed, not ignored.
answeredMessages.release(ctx);
throw err;
}
};
}

export function registerAiRoutes(dispatch: CallbackDispatch, deps: CallbackRouteDeps): void {
const { onAiButtonClick, agentContinuation, scenePauseDeps } = deps.opts;

Expand All @@ -112,7 +276,7 @@ export function registerAiRoutes(dispatch: CallbackDispatch, deps: CallbackRoute
try {
await continueWithAgent(
user,
`Confirmed recipient Telegram ID ${approved.recipientId} for event ${approved.eventId}. Send that invitation with force=true; do not change the ID or use a conflicting username.`,
`Confirmed recipient Telegram ID ${approved.recipientId} for event ${approved.eventId}. Send that invitation with force=true${approved.ownerId === undefined ? '' : ` and owner_id ${approved.ownerId}`}; do not change the ID or use a conflicting username.`,
agentContinuation,
);
finishRecipientApproval(payload, false);
Expand Down Expand Up @@ -189,36 +353,6 @@ export function registerAiRoutes(dispatch: CallbackDispatch, deps: CallbackRoute
});

// AI ask_user button responses — trigger AI continuation
dispatch.set('ai_btn', async (ctx, _payload, _parts, user, data) => {
const firstColon = data.indexOf(':');
const { answerText, restrictedToUserId } = parseAiBtnPayload(data.slice(firstColon + 1), isGroup(ctx));
const lang = (user.language ?? 'en') as Lang;

// In groups, only the user who triggered the question can answer
const clickerId = ctx.from.id;
if (restrictedToUserId !== undefined && clickerId !== restrictedToUserId) {
await ctx.answer({ text: t(lang).callbackErrors.notYourQuestion, show_alert: false });
return;
}

await ctx.answer();
// Keep the question: after a destructive confirmation it is the user's only record of
// what they agreed to. Appending at the end leaves the entity offsets valid.
const question = ctx.message?.text ?? '';
// A later tap from a client that still shows the buttons finds an answer already recorded,
// the same option or another one: the AI must not get a second answer (a destructive
// confirmation would run again, or run after the user declined it).
if (/(?:^|\n\n)✅ [^\n]*$/.test(question)) {
cmdLogger.info({ userId: user.telegram_id }, 'ai_btn tap on an already answered question ignored');
return;
}
const recorded = await recordAskUserAnswer(ctx, question, answerText);
if (!recorded) return;
const cbChatId = ctx.chatId;
if (onAiButtonClick && cbChatId) {
onAiButtonClick(user.telegram_id, cbChatId, answerText).catch((e) => {
cmdLogger.error({ err: e }, 'AI button continuation failed');
});
}
});
const handleTap = createAskUserTapHandler(onAiButtonClick);
for (const action of [AI_BUTTON_ACTION, AI_OPTION_ACTION]) dispatch.set(action, handleTap);
}
Loading
Loading