Skip to content

GetLogEvents implements half of one of its two published pagination tokens, so the documented termination rule cannot be followed #1223

Description

@scttfrdmn

GetLogEvents publishes a pair of directional pagination tokens and substrate implements half of
one of them, so a caller that follows the documented termination rule cannot terminate.

What the page publishes

API_GetLogEvents publishes two response elements, both Length Constraints: Minimum length of 1:

  • nextBackwardToken"The token for the next set of items in the backward direction. The token
    expires after 24 hours. This token is not null. If you have reached the end of the stream, it
    returns the same token you passed in."
  • nextForwardToken"The token for the next set of items in the forward direction. The token
    expires after 24 hours. If you have reached the end of the stream, it returns the same token you
    passed in."

And, in the operation's own overview:

GetLogEvents is a paginated operation. … As long as the nextBackwardToken or nextForwardToken
returned is NOT equal to the nextToken that you passed into the API call, there might be more log
events available. The token that you use depends on the direction you want to move in along the log
stream. The returned tokens are never null.

The request parameter startFromHead also publishes a rule that depends on the pair: "If you are
using a previous nextForwardToken value as the nextToken in this operation, you must specify
true for startFromHead."

What substrate does

getLogEvents (emulator/cloudwatchlogs_plugin.go) declares both members omitempty, never sets
nextBackwardToken at all, and sets nextForwardToken only when a further page exists. So:

  1. nextBackwardToken is always absent. A caller reading the member AWS says "is not null"
    reads nothing.
  2. At the end of the stream both are absent, where AWS returns the same token you passed in.
  3. Therefore the published termination rule is unusable against substrate: a loop written as
    "stop when the returned token equals the one I sent" never sees a returned token, so it either
    compares against an empty string on the first call and stops immediately, or spins. A caller has
    to use the empty-token rule instead, which is what the other three Logs paginators publish and
    this one does not.
  4. startFromHead is not read at all, so the direction the pair exists to express is unmodelled.

This is not a token-validity defect. #1086 converted all four Logs paginators to refuse a
nextToken substrate could not have issued, and recorded this as explicitly out of its scope so the
conversion would not be read as having fixed it — see emulator/cloudwatchlogs_pagination.go and the
docs/services.md section "CloudWatch Logs carried one block four times".

The second half: the published 24-hour expiry

All four Logs paginators publish "The token expires after 24 hours." on the response member, and
no page publishes a code for presenting an expired token — the same shape as EventBridge's
InvalidToken finding with the conclusion missing. Substrate's tokens never expire. Modelling it is
squarely in scope per CLAUDE.md (the simulated clock, not wall time, would judge it) but needs a
code decision #1086 deliberately did not take: the only code all four pages publish for a bad
parameter is InvalidParameterException/400, and attributing expiry to it is a reading, not an
attribution.

Scope this issue to the token pair; the expiry can ship with it or separately, but the decision
should be recorded either way.

Acceptance criteria

  • nextForwardToken is present on every successful GetLogEvents response, including the last
    page, and at the end of the stream equals the nextToken the caller passed in (or the
    first-page token when none was passed).
  • nextBackwardToken is present on every successful response and is never empty.
  • The two tokens are distinguishable, so a token issued as backward is not silently accepted as a
    forward offset (and vice versa) — the Fifteen offset-token sites still answer page one for a token substrate never issued #1086 refusal must keep working, and a directional token
    that decodes cleanly in the wrong direction is the same class of silent wrong answer.
  • startFromHead is read, or its absence is recorded in the plugin as a deliberate divergence
    with the reason.
  • A test walks a stream forward to the end and asserts the documented termination condition — the
    returned token equalling the one passed in — holds, driven entirely through the wire.
  • A decision on the published 24-hour expiry is recorded: either modelled against the simulated
    clock with the code choice argued, or declined in docs/services.md with the reason.
  • docs/services.md's CloudWatch Logs section stops listing the missing backward token among the
    divergences Fifteen offset-token sites still answer page one for a token substrate never issued #1086 left in place.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions