Skip to content

Fix: resolve unexpanded env vars in Slack webhook token#7

Merged
crarau merged 1 commit into
mainfrom
fix/slack-token-env-expansion
Jul 2, 2026
Merged

Fix: resolve unexpanded env vars in Slack webhook token#7
crarau merged 1 commit into
mainfrom
fix/slack-token-env-expansion

Conversation

@crarau

@crarau crarau commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Root Cause

The trigger config in config.ts caches env-var-expanded values via expandEnvVars(). When the cache is populated before the env var is available in process.env (PM2 environment mismatch, startup race condition), the literal string ${ELI_SLACK_BOT_TOKEN} is stored as the token value instead of the real xoxb-... token. This unexpanded string is then sent to Slack as a Bearer token, which returns invalid_auth with no further detail.

This has been silently failing for all Eli channel failure notices since April 29 (every single Slack thread reply to Eli channels has failed).

Fix

  • Add resolveSlackToken() in src/lib/webhooks/resolve-slack-token.ts that detects ${VAR} patterns at use time and re-expands from process.env
  • Import and use it in the webhook handler instead of raw trigger.slackBotToken || process.env.SLACK_BOT_TOKEN
  • Improve error logging to include trigger name and token prefix for faster diagnosis

Alert Reference

Triggered by production alert at 2026-06-25T14:20:02.065Z:
[Slack Webhook] Slack reply failed: invalid_auth thread_ts=1782396209.882939

Testing

  • 7 new unit tests for resolveSlackToken covering: normal tokens, fallback env, unexpanded vars with/without env resolution, custom fallback keys
  • Full test suite passes (186 tests, 17 files)
  • TypeScript check passes (only pre-existing errors in unrelated test file)
  • Confirmed manually that sending literal ${ELI_SLACK_BOT_TOKEN} to Slack returns invalid_auth, while the expanded token works

The trigger config caches env-var-expanded values via expandEnvVars(),
but when the cache is populated before the env var is available (PM2
env mismatch, startup race), the literal string "${ELI_SLACK_BOT_TOKEN}"
is sent as the Bearer token instead of the real value. Slack returns
invalid_auth for that literal string with no further detail, which has
been silently failing for all Eli channel failure notices since April 29.

Add resolveSlackToken() that detects unexpanded ${VAR} patterns at use
time and re-expands from process.env. Also improve error logging to
include trigger name and token prefix for faster diagnosis.
@crarau
crarau merged commit 94c2d72 into main Jul 2, 2026
2 checks passed
@crarau
crarau deleted the fix/slack-token-env-expansion branch July 2, 2026 17:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant