[CI] Parity: surface XML failure messages in the summary#3364
Merged
jithunnair-amd merged 4 commits intoJun 24, 2026
Conversation
Add the per-test error message (parsed from the test report XML, already stored in the message_<set> CSV column) to the parity summary: - CSV FAILED TESTS table gets a new 'Error Message' column so downstream dashboard parsing has it in structured form. - Markdown summary gets a top-level collapsible 'Failure messages' <details> with one nested <details> per failed test (full traceback in an HTML-escaped <pre>), so the message is available without bloating the table.
The full .md is piped into $GITHUB_STEP_SUMMARY (1 MB cap); many full tracebacks can push it over the limit and GitHub drops the whole summary. Truncate each message to its tail (where the exception is) in the md; the CSV 'Error Message' column keeps the full text for the dashboard.
Replace the separate bottom 'Failure messages' section with an 'Error Message' column in the FAILED TESTS markdown table. Each cell is a <details><summary>view</summary> toggle wrapping a <pre> (newlines as , pipes escaped) so the table stays scannable and expands per row. The CSV 'Error Message' column (full text) is unchanged.
…y cap The per-row 'Error Message' column hard-truncated each message to 2000 chars so the markdown couldn't exceed GitHub's per-step $GITHUB_STEP_SUMMARY limit (1 MiB), which clipped useful tracebacks even on small runs with room to spare. Raise the per-message cap to 50k and size messages against a global 950 KB budget (1 MiB minus headroom for the surrounding tables): the rest of the summary is measured first, then the largest uniform per-message cap that fits the remaining budget is applied. Typical runs render full tracebacks; only when a run would exceed the cap do the longest (outlier) messages get clipped. Full text always remains in the CSV 'Error Message' column. https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-commands#step-isolation-and-limits
|
Jenkins build for 9401da6c7e37cc3b9308db24b906c1fba2166cb6 commit is in progress |
pablo-garay
reviewed
Jun 23, 2026
Comment on lines
880
to
889
| if has_set2: | ||
| line += f" | {t.get(f'shard_{s2_name}', '')}" | ||
| line += f" | {_xml_test_shard(t, s2_name)}" | ||
| line += f" | {t[f'status_{s1_name}']}" | ||
| if has_set2: | ||
| line += f" | {t.get(f'status_{s2_name}', '')}" | ||
| line += f" | {t.get('also_failing_in', '')}" | ||
| line += f" | {_job_id_link(t.get(f'job_url_{s1_name}', ''))}" | ||
| if has_set2: | ||
| line += f" | {_job_id_link(t.get(f'job_url_{s2_name}', ''))}" |
There was a problem hiding this comment.
nit : not in this PR change diff but this could be (consolidated in) one "if has_set2:" condition followed by the lines after
pablo-garay
approved these changes
Jun 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Split out of #3210 (3 of 3) for easier review.
Surfaces test failure messages inline in the parity summary (
generate_summary.py):Independent of the other two split PRs (touches only
generate_summary.py).Made with Cursor