v11/CCK v22 conformance

[luke-hill]

Description

Currently cucumber is v20 of the CCK conformant.

First step is to make it v22 conformant. Which requires adding in a bunch of new scenarios to the CCK testing, and then validating which items are still passing and what changes need to be made

Type of change

Please delete options that are not relevant.

• Refactoring (improvements to code design or tooling that don't change behaviour)
• New feature (non-breaking change which adds new behaviour)
• Breaking change (will cause existing functionality to not
  work as expected)

Checklist:

Current passing state 75/93

Items to check / fix (First pass)

• Reversed order being executable from the CLI
• When a global hook fails. It seems to kill the entire process and not capture any messages
• New assertion for consistent test run id does not seem to be always present#
• Attachment is missing timestamp
• TestStepResult is missing some keys
• The retry scenario is failing for an unknown reason

[luke-hill]

Posting in here so there's a visible update outside of code

# CCK v22 conformance status update - Jan 2026
  # OVERALL: 93 examples, 15 failures, 78 passed
  # SANITIZED: 69 examples, 0 failures, 69 passed

[luke-hill]

ping @brasmusson after all merges, re-analysis and tidy up. I found one new issue (Which will be auto-fixed once we are testing against CCK v29 - we're ahead of the game). I have now re-enabled a bunch of the previously failing CCK scenarios and we now have only 10 left to fix.

[luke-hill]

JRuby issue requires a major bump (Given the next version is going to be a major - I'm going to do a bump in a new PR and then fold this in. Small ticket.

[luke-hill]

@olleolleolle One example here (And below in comparator), of non-conformance items tripping things up because they're very out of sync.

TL;DR - I'm pursuing CCK v22 conformance which is out of sync with other bits

[luke-hill]

JRuby being fixed in #1824