Skip to content

[FLINK-40530][runtime-web] Isolate per-entry failures in job overview aggregation - #29062

Open
argoyal2212 wants to merge 1 commit into
apache:masterfrom
argoyal2212:argoyal/fhs-overview-per-entry-isolation
Open

[FLINK-40530][runtime-web] Isolate per-entry failures in job overview aggregation#29062
argoyal2212 wants to merge 1 commit into
apache:masterfrom
argoyal2212:argoyal/fhs-overview-per-entry-isolation

Conversation

@argoyal2212

Copy link
Copy Markdown
Contributor

What is the purpose of the change

HistoryServerArchiveFetcher#updateJobOverview() aggregates every archived job's overview JSON into one combined /jobs/overview response. Today a single malformed or version-incompatible archive throws during aggregation and aborts the whole update, silently returning empty/stale overview data for every job, not just the offending one. This PR adds per-entry isolation so one bad archive can no longer take down the entire job listing.

Brief change log

  • updateJobOverview() now catches and skips a single archive's parse failure instead of letting it abort the whole aggregation
  • Logs a warning per skipped entry plus a summary count, instead of one opaque top-level error
  • Guards against overwriting a previously-good combined overview if every entry fails on a given refresh cycle

Verifying this change

This change added tests and can be verified as follows:

  • Added testUpdateJobOverviewSkipsMalformedEntryInsteadOfFailingEverything, which injects one malformed per-job overview entry alongside a valid one and asserts the valid job is still listed
  • Added testUpdateJobOverviewDoesNotWipeGoodOverviewWhenAllEntriesAreMalformed, which corrupts the only entry and asserts the previously written combined overview is preserved rather than replaced with an empty one

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? not applicable

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: GitHub Copilot CLI

… aggregation

A single malformed archive no longer aborts the whole /jobs/overview update; it is skipped and logged, and a good overview is kept if all entries fail.
@argoyal2212

Copy link
Copy Markdown
Contributor Author

@zentol @gaborgsomogyi @dmvk Can I please get some help on this PR review. Thanks!

@flinkbot

flinkbot commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants