Repository navigation
fix(smoke): GNOME 51 audit verdicts for rows 4, 5, 6 and 10; notification dismissal check could not fail - #1209
Conversation
…nd 10 Assisted-by: Claude Opus 5.5 via Claude Code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WcWm7qQa538Zjcuvv8jBtn Signed-off-by: Danathar <Danathar@users.noreply.github.com>
…ell's Assisted-by: Claude Opus 5.5 via Claude Code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WcWm7qQa538Zjcuvv8jBtn Signed-off-by: Danathar <Danathar@users.noreply.github.com>
"Notification banner is no longer showing via Shell.Eval" also required Main.messageTray._bannerBin to be hidden. On GNOME 51 the bin stays visible with no banner in it (#827 probe 3: _banner is null 0.2 s after CloseNotification while _bannerBin.visible stays true for 20 s), so the check never passed; the step then force-destroyed the banner and, failing that, only printed a warning. It could not fail, on GNOME 50 or 51, whether or not CloseNotification worked. Check only the banner (null, undefined or hidden, which also covers GNOME 50 keeping a hidden ref), keep the 20 s allowance for slow QEMU banners, and fail with the last Shell.Eval output instead of force-dismissing. Assisted-by: Claude Opus 5.5 via Claude Code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WcWm7qQa538Zjcuvv8jBtn Signed-off-by: Danathar <Danathar@users.noreply.github.com>
…e smoke scenario does Assisted-by: Claude Opus 5.5 via Claude Code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WcWm7qQa538Zjcuvv8jBtn Signed-off-by: Danathar <Danathar@users.noreply.github.com>
…rents Assisted-by: Claude Opus 5.5 via Claude Code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WcWm7qQa538Zjcuvv8jBtn Signed-off-by: Danathar <Danathar@users.noreply.github.com>
…r step's search Assisted-by: Claude Opus 5.5 via Claude Code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WcWm7qQa538Zjcuvv8jBtn Signed-off-by: Danathar <Danathar@users.noreply.github.com>
…ner shows Probe 3 (#827 row 10) showed why the stricter check failed: after CloseNotification our notification leaves the message tray at once, but the visible banner is an earlier, unrelated one ("Settings") that GNOME Shell holds on screen while the session is idle; a headless VM is always idle (idle time ~4 min), and one keypress clears it in ~2 s. The old "no banner at all" check therefore tracked the wrong notification. Check that "Test Notification" is neither the visible banner nor queued in the tray, using one shared title constant for the send step. Assisted-by: Claude Opus 5.5 via Claude Code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WcWm7qQa538Zjcuvv8jBtn Signed-off-by: Danathar <Danathar@users.noreply.github.com>
…smoke-a Assisted-by: Claude Opus 5.5 via Claude Code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WcWm7qQa538Zjcuvv8jBtn Signed-off-by: Danathar <Danathar@users.noreply.github.com>
…sal check Smoke-a runs 43554149449/43554157456 showed the tray holding several "Test Notification" entries: other scenarios and retries send the same title and never close theirs, so a title match could not tell whether this scenario's notification was dismissed (CloseNotification did remove one). Each send now adds a short unique suffix, stored on the context, and the dismissal check looks for that title only. Assisted-by: Claude Opus 5.5 via Claude Code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WcWm7qQa538Zjcuvv8jBtn Signed-off-by: Danathar <Danathar@users.noreply.github.com>
… GNOME 51 skill section; drop probe 3 Probe 3 (removed here) measured each workaround's guarded property on the pre-workaround path, on dakota:testing and utah:testing (GNOME Shell 51.1): - Row 4: Files, Settings and Firefox expose a populated `frame` toplevel; no `filler` occurred. Obsolete on 51 for every app measured. - Row 5: the sidebar step's only match is a `list item` "Open Personal Folder" (the pre-GNOME 50 role). Obsolete on 51. - Row 6: Files registers as `org.gnome.Nautilus`. Still required. - Row 10: broken on 51 and fixed earlier in this branch. - qecore 4.19.3 (with dogtail 2.0.3, behave 1.3.3) is what the GNOME 51 runs use, and works there; releases since name no GNOME 51 change. The Calculator and Text Editor divergence write-up moves to its own reference (gnome-51-app-divergences.md) to keep the audit under 200 lines. SKILL.md's two GNOME 51 audit sections become one GNOME 51 section. Assisted-by: Claude Opus 5.5 via Claude Code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WcWm7qQa538Zjcuvv8jBtn Signed-off-by: Danathar <Danathar@users.noreply.github.com>
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
…as dismissed Fresh-context review of #1209 found two ways the dismissal check could still pass without proving anything: - banner_gone_js caught a tray-read error as `queued = false`, so an API change in GNOME Shell would read as "gone". The JS now reports the error, and any error, failed Shell.Eval or unparseable reply counts as not gone. - With no send step the title defaulted to the bare "Test Notification", which nothing sends any more, so the check always passed. It now fails unless a send recorded a title. The close step also requires Shell to hold the notification (queued or shown) before closing it, so the check can only credit CloseNotification with removing something that was there. Unit tests now drive the step with Shell.Eval replies encoded as Shell sends them: an error reply, a `(false, '')` reply, no send, not seen, and a close refused when the tray cannot be read. Assisted-by: Claude Opus 5.5 via Claude Code Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WcWm7qQa538Zjcuvv8jBtn Signed-off-by: Danathar <Danathar@users.noreply.github.com>
|
@Danathar this PR is back in your court ( Failing checks:
Reply here or push a new head to this branch (same sign-off); CI and the review swarm re-run automatically and the PR returns to the normal path. Your relay session received this as a |
|
The failing
I didn't add an allowlist entry here. utah#1198 marks that as a separate testsuite decision, and 🐝 Hive Agent: 🐝 Hive Agent: |
main's #1209 measured the verdicts for rows 4, 5, 6 and 10 and rewrote the notification dismissal check, so this merge takes main's gnome-51-audit.md, gnome_notifications_steps.py and its unit tests. The row 10 log lines are dropped because the branches they reported no longer exist. The rows 4, 5 and 6 readings in gnome_files_steps.py stay, and the SKILL.md note now says what they re-check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Danathar <Danathar@users.noreply.github.com>
Summary: Every GNOME 50 workaround in the #827 audit now has a GNOME 51 verdict from a targeted probe, and the one that was broken is fixed. The notification-dismissal check could never fail. On GNOME 51 it required a widget to hide that never hides, and it treated any visible banner as ours, so it always ended in a forced dismissal and a warning. It now checks that this scenario's own notification is gone. qecore 4.19.3 is confirmed working on GNOME 51, and the GNOME skill doc gets one GNOME 51 section.
Refs #827. What's left is below.
Verdicts (GNOME Shell 51.1,
dakota:testingandutah:testing)filler/frametoplevelsframe, neverfiller. Calculator, Text Editor and Clocks ship on neither imagebuttonrolelist item"Open Personal Folder", the pre-GNOME 50 roleorg.gnome.Nautilus, notnautilusRows 1–3, 7–9, 11 and 12 were settled in #1199 and #1206.
The notification fix
The probe showed what the old check was measuring:
CloseNotification, this scenario's notification leaves the tray.<Shift>press clears it within 2.3 s._bannerBinstays visible on 51 even with no banner in it.The step now gives each sent notification a unique title. It passes when that notification is neither the visible banner nor in the tray, and it fails with the details instead of force-dismissing.
After a fresh-context review (b21b5bb), two further gaps are closed:
Shell.Evalor an unparseable reply never counts as gone. The JS reports the error instead of treating it as "not queued".The fixed scenario passes every step on both images at b21b5bb: 43559017193 (dakota) and 43559025260 (utah: 0 scenarios failed, and the job fails only at the SELinux gate on utah#1198).
Not tested on GNOME 50: no tested image runs GNOME 50, so CI can't exercise the GNOME 50 path. The check needs only
Main.messageTray._banner,getSources()and each notification'stitle.qecore on GNOME 51
The GNOME 51 runs use qecore 4.19.3, dogtail 2.0.3 and behave 1.3.3, read from the test process, and smoke-a, smoke-b and the GNOME OS canary pass. qecore's later releases (4.20–4.22.1) name no GNOME 51 change. 4.22's missing-keyring handling matches row 12 on utah.
Docs
gnome-51-audit.md: rows 4, 5, 6 and 10; a "round 3" probe section; a qecore note; and a summary saying every row now has a verdict.gnome-51-app-divergences.md, which keeps the audit under 200 lines.SKILL.md: its two GNOME 51 audit sections become one GNOME 51 section, at 499 lines.Tests
Shell.Evalreplies encoded as Shell sends them. They cover: dismissed, still shown, a tray read error, a(false, '')reply, no send, not seen before close, a close refused when the tray can't be read, and unique titles per send.test_quarantine_age_git.pyfailures that also fail on main. Docs validation and the behave dry run pass.Left on #827
Ctrl+N/Ctrl+Qdivergences seen on plain GNOME OS 51 are documented, not fixed. Neither app ships ondakota:testingorutah:testing.🤖 Generated with Claude Code
https://claude.ai/code/session_01WcWm7qQa538Zjcuvv8jBtn