Skip to content

fix: stabilize LINE authentication, identity resolution, messaging and tests - #297

Open
meramsey wants to merge 38 commits into
beeper:mainfrom
meramsey:upstream/line-stabilization-20261029
Open

meramsey wants to merge 38 commits into
beeper:mainfrom
meramsey:upstream/line-stabilization-20261029

Conversation

@meramsey

@meramsey meramsey commented Oct 30, 2026 •

Copy link
Copy Markdown

Summary and reviewer guide

This consolidates public LINE ID/DM resolution, friendly login methods, QR and
authentication error/cooldown safety, send acknowledgements/cancellation, inbound
media integrity/metadata, and supporting test/CI/build work.

Review documents (including exact commands, expected results, evidence and risks):

The files also link to one another and to implementation/tests using relative
paths, so they remain navigable in a checkout.

Important current-base limitation

The implementation comparison is against the last integrated upstream base
45a48b9. On 2026-11-02, main is 10e0a5f; GitHub reports conflicts, with a
content conflict in pkg/connector/client.go confirmed by git merge-tree.
New upstream reconnect/error/media/avatar/sticker improvements must be retained
when integrating or splitting. This documentation update does not resolve the
runtime conflict or claim a tested merge with today's main.

Short before/after

Area Before (integrated upstream base) After this PR
Login UX Framework flow IDs; short qr/email/help rejected Friendly login, login help, login qr, login email; full IDs and QR-disabled behavior retained
QR/auth/reconnect Concurrent QR operations and generic/sparsely classified failures; no shared login gate Serialized lifecycle, structured stages, selectively redacted safe provider messages, shared login claims/cooldowns
Rate limits No connector-wide local login wait Handle 429, RETRY_LATER, temporary verification unavailability and Retry-After; stored-credential path respects waits
Public ID/DM Input treated as MID; portal persistence before validated lookup Resolve public IDs, validate canonical/opaque IDs, preserve gateway case, reuse/reopen canonical portals
Sending Context/error/empty-ack handling could misreport outcomes Request cancellation reaches send; provider codes retained; require a LINE ack ID; no new exactly-once guarantee
Media Encrypted-media MAC stripped unchecked; images presented as JPEG Authenticate before plaintext, video-specific MAC; detect image MIME/name/size/dimensions where supported
Tests/CI/build No new test modes/floors; redundant heavy fixtures Bounded deterministic fixtures, normal/race/coverage CI and caches, coverage floors, fixed-epoch explicit-source builds

See the full comparison for observable effects and inherited upstream behavior:
for example, preview-image upload and #296's search cap are already in the base,
not new features invented here. Sync policy is not changed to add bulk syncing.

Root causes and implemented fixes

  • Register the LINE login wrapper in LineConnector.Init; mautrix-go's original
    parser accepts only full flow IDs. Aliases delegate to its existing flow and
    retain framework permissions/command-state handling.
  • Validate public-ID/MID lookup results before persisting portals, preserve
    opaque case-sensitive gateway identities, reject mismatched/ambiguous contacts,
    and keep lookup-only distinct from chat creation.
  • Propagate QR-create/verification/token and session-establishment stages;
    retain meaningful provider wording only after sanitization; enforce shared
    claims/waits on interactive and stored-credential login paths.
  • Pass Matrix cancellation to the send RPC, reject missing acknowledgements,
    and retain provider errors for existing auth-recovery behavior.
  • Verify encrypted media HMAC before decryption and attach detected image
    metadata while retaining current integrated upstream media handling.

Testing: what is and is not evidence

  • Full normal/race/coverage/release reports apply to integrated runtime source
    07badf92460e5ae744b90d1c0c0a46cc30396218. Later commits through the prior PR
    head 81f5021 changed documentation only: runtime/tests/CI are identical.
  • Each normal/race/coverage suite: 512 passing test/subtest events, 1 existing
    skip, 0 failures
    ; coverage gates passed. Release also passed format, vet,
    staticcheck, module verification, coverage-checker tests and build. The skip is
    the old JSON-IPC vector replay awaiting a typed-runner rewrite.
  • Fresh targeted replay on 81f5021 for this review: 146 pass events, no skips
    or failures
    , plus fast-mode format/vet/staticcheck on selected packages.
  • Fixed-epoch Linux amd64 builds of checkpoint f36c646b with Go 1.26.10 and
    libolm 3.2.16 reproduced byte-for-byte. This is not a fresh build or deployment
    of the PR's merged runtime.
  • Historical live startup/appservice/database verification is attributable to
    v1.2.0+dev.f36c646b. The author's earlier report of use over multiple days lacks
    a per-scenario/source inventory; it is not marked live pass for each item.
  • Automated tests use local crypto/fixtures, temporary databases and mocked
    LINE/Matrix responses. They do not prove live LINE compatibility. GitHub checks
    are currently unreported, not passed; current-main integration is untested.

The test matrix
labels AUTOMATED PASS, LIVE VERIFIED, MANUAL NOT RUN, provider-condition-only
tests, and KNOWN LIMITATION separately. It starts with startup/help, then optional
one-time QR login, known-contact lookup/reuse, one message each way, benign media,
and reconnect/log review. Use a disposable account; do not provoke rate limits
or repeatedly log out/generate QR codes.

Blast radius and compatibility

Highest-risk paths are authentication/reconnect, identity/recipient selection,
send acknowledgements and authenticated media. Cooldowns are in-memory and
keyed by Matrix user ID
, not durable or globally keyed by LINE account. Default
waits are 15/5/10 minutes for rate limit/retry-later/verification-unavailable;
Retry-After may extend them. These are bridge waits, not measured LINE restriction
durations or protection against restrictions.

No PR-specific SQL/schema migration, framework dependency bump, new required
configuration, or UserLoginMetadata field change is introduced. Existing login
finalization still writes credentials/session/E2EE metadata; portal creation
still writes ghosts/rooms/membership. Restoring a binary does not undo remote
messages, new sessions or portals. See the complete assessment for rollback,
network-frequency effects and failure modes. Existing crypto devices/encryption
are not reset or weakened.

Relationship, attribution and smaller review units

  • Intended replacement/superset of open #271:
    all nine commits are represented by patch ID. Adds later auth/identity/
    validation/CI/build corrections. fix: resolve LINE IDs and harden sends, QR login, and media #271 remains open; please compare before
    merging overlapping versions independently.
  • Adapts compatible work from the six-commit
    fork optimization PR #1: two identical
    patches, later adaptations of modes/coverage/cache tooling. Environment-specific
    historical benchmark report not copied. Its branch/base remain unchanged.
  • Media authentication and metadata retain credit to Adrien Vermersch's
    #196 and
    #189.
  • The reviewer requested smaller chunks. The blast-radius document proposes
    identity/DM, auth/reconnect, messaging/media, and test/CI/build units, with
    shared fixture/dependency considerations. These are proposed, not already
    published split PRs. No existing PR is closed, retargeted or merged here.

Known limitations

  • No complete live multi-account, restriction-duration, network interruption,
    all-media/group/LS-mode or Matrix-client matrix. No exactly-once delivery claim
    for ambiguous send timeouts; verify with the recipient before retrying.
  • The Matrix puppet/device ownership E2EE warning remains unresolved; no fix
    claimed. Do not reset devices or disable encryption to hide it.
  • Historical optional lookup-shape diagnostics remain deferred for schema/privacy
    review; current safe validation/error handling remains.
  • No live account action, deployment or service restart was performed for this
    reviewer-documentation update. Conflict resolution and smaller PR publication
    remain follow-up work.
  • Documentation audit found a CI prerequisite gap: the normal test job's alias
    step invokes fast-mode staticcheck without installing it in that job. Local
    validation had it on PATH; no GitHub check failure is available. This needs a
    focused follow-up, not a claim of green CI.

meramsey and others added 30 commits October 28, 2026 05:32
Adapt beeper#196 (a6d06b0) to the shared media decryptor, covering both uploads and direct media. Preserve upstream fail-closed key fallback and route video to chunk-hash HMAC verification. Add valid, tampered and wrong-mode coverage for all media kinds.

Co-authored-by: Adrien Vermersch <adrien.vermersch@devolab.net>
Select the image metadata changes from beeper#189 at a4d8c11, retaining current download, size-limit and authentication handling. Keep the separate 60-retry OBS policy out of this scoped fix. Includes bounded ISO-BMFF brand detection and regression tests.

Co-authored-by: Adrien Vermersch <adrien.vermersch@devolab.net>
Replace 6000 expensive eviction cache misses with three complete cache rotations, per-call victim and LRU assertions, and explicit refresh-boundary coverage. Preserve the separate 6000-call reuse and real heap-exhaustion tests. Give cold signing a separate setup budget while retaining the one-second cancellation bound; cancel and join the request worker on every exit path.
Share exact identifier lookup with search, validate canonical user MIDs and reuse returned contact data. Delay portal persistence until remote resolution succeeds, honor lookup-only requests, and pass complete DM information to bridgev2. Add cancellable lookup RPCs and isolated database regressions for identity consistency, errors, cancellation, concurrent portal reuse and Matrix creation retries.
Reject overlapping Start/Wait calls before they consume polling stages or mutate authentication state. Preserve caller and attempt expiry instead of reporting every canceled start as context.Canceled. Reproduce both defects and cover mocked QR presentation, scan/PIN approval, token verification, missing-keychain persistence rejection, expiry and cancellation.
Add a typed AuthError model in pkg/line that normalizes LINE provider
exceptions into stable categories. QR RPC errors now carry a structured
AuthError; the connector uses it to choose user-facing messages, apply
per-account cooldowns, and emit sanitized log fields.

- Introduce AuthErrorCategory/AuthErrorStage enums and AuthError struct.
- Build AuthError from qrRPCError with safe provider-reason/alert handling.
- Export QRRPCError alias so callers can access AuthError().
- Map SecondaryQrCodeException codes and TalkException codes to categories.
- Classify email login errors into the same categories.
- Add 10-minute cooldown for verification-temporarily-unavailable errors.
- Log structured auth_error_* fields for QR failures.
- Prevent raw provider reason strings from leaking into user-facing email
  login messages.
- Add regression tests for classification, cooldowns, and provider-text
  redaction.
- Make build.sh honor SOURCE_DATE_EPOCH for reproducible builds.
- Update QR_REGRESSION_AUDIT.md with structured-error and cooldown docs.
Allow build.sh to be used with -o and other go build flags that must
appear before the package argument.
Expose the existing native malloc/free alongside ModuleMem for bounded resource fixtures. Prefill the real signing heap, then retain the original real key-allocation abort, recovery, signature and E2EE-isolation assertions. Keep the 6000-signature and 1800-decryption stress loops unchanged. Add allocation isolation, exhaustion and coalescing/reuse tests; generated crypto and default runtime behavior are unchanged.
Use synctest quiescence to check the album-preview worker limit without scheduler-dependent sleeping. Keep ordering and concurrency assertions and bounded failure guards. Write crypto vectors to t.TempDir with mandatory write checks and test-scoped logging, enabling safe parallel execution of the two independent vector tests.
Pass the Matrix request context to sendMessage, preserve structured TalkException details for existing recovery logic, and reject missing acknowledgements instead of returning success or dereferencing nil. Add correlated send-stage logs without message bodies and regressions for ownership, canonical recipients, API failures and cancellation. These defects are reproduced locally; the reported live Element delivery failure still needs event-level evidence.
Verify public-ID reopening after leaving a Matrix room, rejection of mismatched MIDs and expired lookup requests, and mandatory fresh approval for passwordless QR sessions after token recovery is unavailable.
Label existing connect, LINE fullSync and post-backfill synchronization triggers without changing policy. Document canonical DM resolution, retained retryable portal rows, outbound stage/account diagnostics, independent QR lifecycle findings and live-verification boundaries.
Label connection, LINE fullSync and historical-state reconciliation without changing sync behavior. Document canonical identity resolution, portal retry/reopening, outbound correlation and ownership checks, QR lifecycle fixes, mocked coverage and remaining live-verification requirements.
@meramsey

Copy link
Copy Markdown
Author

@clins1994 any chance this could get reviewed and possibly merged ?

Was tested over multiple days different times and hoping to get the improvements up streamed before next official release is made so I don't have to rely on my fork ideally.

@clins1994

clins1994 commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

@meramsey hi 👋 thanks for the pr. first off i can't really merge prs. that's on the beeper team. this said i can help you with a review or testing 👀

feedback: (a) the before/after isn't super clear to me nor how i can test this thoroughly. may i ask for a before/after comparison (table or diagram) and a list of test items you did or did not do that way i can replay them on my bridge? (b) and blast radius?

(c) could you split it in smaller chunks?

@meramsey

meramsey commented Nov 3, 2026

Copy link
Copy Markdown
Author

@clins1994 Thanks for taking the time to review this and offering to help test it! Understood about merging being up to the Beeper team. You're right: the scope grew, and I hadn't made either the behavior changes or the test evidence clear enough.

I've pushed documentation in 4971ee3 and linked it from the PR description:

The matrix separates automated passes from attributable live observations and manual/provider-condition scenarios not run. The previous full suites passed on the integrated runtime source (512 pass events and one existing skip per suite); later changes were documentation-only. A fresh targeted run on the pre-documentation head passed 146 events. That is not evidence that every scenario works against live LINE. The broad multi-day live-use report doesn't provide a per-scenario/source inventory, so I've kept those live checks unverified. The guide starts with startup/help and known-contact lookup; QR only if needed, using a disposable account—not repeated logouts or attempts to trigger restrictions.

Highest-risk changes are authentication/reconnect, recipient resolution, outbound acknowledgements and authenticated media. In particular, cooldowns are memory-only and keyed by Matrix user, and cancellation/ack handling does not guarantee exactly-once delivery after a network timeout. No PR-specific schema migration is added, but new sessions/portals/messages aren't undone by a binary rollback. The E2EE puppet/device warning is still unresolved.

On (c), splitting makes sense. I've proposed identity/DM, auth/reconnect, messaging/media, and test/CI/build boundaries with dependencies called out; I haven't created the split PRs yet. Today's main also conflicts in client.go, so that needs reconciliation and fresh validation, not a claim of merge readiness. GitHub currently reports no checks; the docs separately flag a missing staticcheck prerequisite in the test job for follow-up.

Thanks again—feedback on those boundaries or differences you see when replaying the safe scenarios would be especially useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants