Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions .rubocop_todo.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# This configuration was generated by
# `rubocop --auto-gen-config`
# on 2025-12-09 18:02:29 UTC using RuboCop version 1.71.2.
# on 2025-12-15 17:20:38 UTC using RuboCop version 1.71.2.
# The point is for the user to remove these configuration records
# one by one as the offenses are removed from the code base.
# Note that changes in the inspected code, or installation of new
Expand All @@ -11,6 +11,7 @@
# TODO - [LH] -> Jul '24 - 369 files inspected, 661 offenses detected, 98 offenses autocorrectable
# TODO - [LH] -> Jan '25 (Updated deps and v10 prep) - 369 files inspected, 704 offenses detected, 112 offenses autocorrectable
# TODO - [LH] -> Mar '25 (v10 prep) - 370 files inspected, 721 offenses detected, 116 offenses autocorrectable
# TODO - [LH] -> Dec '25 - 375 files inspected, 713 offenses detected, 109 offenses autocorrectable
# TODO - [LH] -> Dec '26 (query prep) - 378 files inspected, 729 offenses detected, 109 offenses autocorrectable

# Offense count: 1
Expand Down Expand Up @@ -56,12 +57,12 @@ Metrics/AbcSize:
# Configuration parameters: CountComments, CountAsOne, AllowedMethods, AllowedPatterns, inherit_mode.
# AllowedMethods: refine
Metrics/BlockLength:
Max: 52
Max: 53

# Offense count: 14
# Configuration parameters: CountComments, CountAsOne.
Metrics/ClassLength:
Max: 515
Max: 516

# Offense count: 10
# Configuration parameters: AllowedMethods, AllowedPatterns.
Expand All @@ -71,7 +72,7 @@ Metrics/CyclomaticComplexity:
# Offense count: 83
# Configuration parameters: CountComments, CountAsOne, AllowedMethods, AllowedPatterns.
Metrics/MethodLength:
Max: 64
Max: 65

# Offense count: 15
# Configuration parameters: CountComments, CountAsOne.
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Please visit [cucumber/CONTRIBUTING.md](https://github.com/cucumber/cucumber/blo

## [Unreleased]
### Added
- Added a new option for running order `--reverse` which will run the scenarios in reverse order ([#1807](https://github.com/cucumber/cucumber-ruby/pull/1807) [luke-hill](https://github.com/luke-hill))
- A first initial iteration of the new `cucumber-query` structure ([#1801](https://github.com/cucumber/cucumber-ruby/pull/1801) [luke-hill](https://github.com/luke-hill))
> This will be used for the migration of all existing formatters - becoming the building blocks for the future of cucumber formatters
> which will begin being migrated in the start of 2026
Expand Down
244 changes: 244 additions & 0 deletions features/docs/cli/ordering.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,244 @@
Feature: Ordering

Cucumber can run scenarios in different orders. By default, scenarios are run in the order they
appear in the feature files. Use the `--order random` switch to run scenarios in random order.

You can also run cucumber in a reverse order using `--order reverse`.

Using different ordering can help you detect situations where you have state
leaking between scenarios, which can cause flickering or fragile tests.

If you do find a random run that exposes dependencies between your tests,
you can reproduce that run by using the seed that's printed at the end of
the test run.

For a given seed, the order of scenarios is constant, i.e. if step A runs
before step B, it will always run before step B even if other steps are
skipped.

Background:
Given a file named "features/bad_practice_part_1.feature" with:
"""
Feature: Bad practice, part 1

Scenario: Set state
Given I set some state

Scenario: Depend on state from a preceding scenario
When I depend on the state
"""
And a file named "features/bad_practice_part_2.feature" with:
"""
Feature: Bad practice, part 2

Scenario: Depend on state from a preceding feature
When I depend on the state
"""
And a file named "features/unrelated.feature" with:
"""
Feature: Unrelated

@skipme
Scenario: Do something unrelated
When I do something
"""
And a file named "features/step_definitions/steps.rb" with:
"""
Given('I set some state') do
$global_state = 'set'
end

Given('I depend on the state') do
raise 'I expect the state to be set!' unless $global_state == 'set'
end

Given('I do something') do
end
"""

Scenario: Run scenarios in order
When I run `cucumber`
Then it should pass
And the stdout should contain exactly:
"""
Feature: Bad practice, part 1

Scenario: Set state # features/bad_practice_part_1.feature:3
Given I set some state # features/step_definitions/steps.rb:1

Scenario: Depend on state from a preceding scenario # features/bad_practice_part_1.feature:6
When I depend on the state # features/step_definitions/steps.rb:5

Feature: Bad practice, part 2

Scenario: Depend on state from a preceding feature # features/bad_practice_part_2.feature:3
When I depend on the state # features/step_definitions/steps.rb:5

Feature: Unrelated

@skipme
Scenario: Do something unrelated # features/unrelated.feature:4
When I do something # features/step_definitions/steps.rb:9

4 scenarios (4 passed)
4 steps (4 passed)
0m0.012s
"""

@global_state
Scenario: Run scenarios randomized
When I run `cucumber --order random:41544 -q`
Then it should fail
And the stdout should contain exactly:
"""
Feature: Bad practice, part 1

Scenario: Depend on state from a preceding scenario
When I depend on the state
I expect the state to be set! (RuntimeError)
./features/step_definitions/steps.rb:6:in `"I depend on the state"'
features/bad_practice_part_1.feature:7:in `I depend on the state'

Feature: Unrelated

@skipme
Scenario: Do something unrelated
When I do something

Feature: Bad practice, part 2

Scenario: Depend on state from a preceding feature
When I depend on the state
I expect the state to be set! (RuntimeError)
./features/step_definitions/steps.rb:6:in `"I depend on the state"'
features/bad_practice_part_2.feature:4:in `I depend on the state'

Feature: Bad practice, part 1

Scenario: Set state
Given I set some state

Failing Scenarios:
cucumber features/bad_practice_part_1.feature:6
cucumber features/bad_practice_part_2.feature:3

4 scenarios (2 failed, 2 passed)
4 steps (2 failed, 2 passed)

Randomized with seed 41544
"""

@force_legacy_loader
Scenario: Rerun scenarios randomized
When I run `cucumber --order random --format summary`
And I rerun the previous command with the same seed
Then the output of both commands should be the same

@global_state
Scenario: Run scenarios randomized with some skipped
When I run `cucumber --tags "not @skipme" --order random:41544 -q`
Then it should fail with exactly:
"""
Feature: Bad practice, part 1

Scenario: Depend on state from a preceding scenario
When I depend on the state
I expect the state to be set! (RuntimeError)
./features/step_definitions/steps.rb:6:in `"I depend on the state"'
features/bad_practice_part_1.feature:7:in `I depend on the state'

Feature: Bad practice, part 2

Scenario: Depend on state from a preceding feature
When I depend on the state
I expect the state to be set! (RuntimeError)
./features/step_definitions/steps.rb:6:in `"I depend on the state"'
features/bad_practice_part_2.feature:4:in `I depend on the state'

Feature: Bad practice, part 1

Scenario: Set state
Given I set some state

Failing Scenarios:
cucumber features/bad_practice_part_1.feature:6
cucumber features/bad_practice_part_2.feature:3

3 scenarios (2 failed, 1 passed)
3 steps (2 failed, 1 passed)

Randomized with seed 41544

"""

@global_state
Scenario: Run scenarios in reverse order
When I run `cucumber --order reverse -q`
Then it should fail
And the stdout should contain exactly:
"""
Feature: Unrelated

@skipme
Scenario: Do something unrelated
When I do something

Feature: Bad practice, part 2

Scenario: Depend on state from a preceding feature
When I depend on the state
I expect the state to be set! (RuntimeError)
./features/step_definitions/steps.rb:6:in `"I depend on the state"'
features/bad_practice_part_2.feature:4:in `I depend on the state'

Feature: Bad practice, part 1

Scenario: Depend on state from a preceding scenario
When I depend on the state
I expect the state to be set! (RuntimeError)
./features/step_definitions/steps.rb:6:in `"I depend on the state"'
features/bad_practice_part_1.feature:7:in `I depend on the state'

Scenario: Set state
Given I set some state

Failing Scenarios:
cucumber features/bad_practice_part_2.feature:3
cucumber features/bad_practice_part_1.feature:6

4 scenarios (2 failed, 2 passed)
4 steps (2 failed, 2 passed)
"""

@global_state
Scenario: Run scenarios in reverse order with some skipped
When I run `cucumber --tags "not @skipme" --order reverse -q`
Then it should fail
And the stdout should contain exactly:
"""
Feature: Bad practice, part 2

Scenario: Depend on state from a preceding feature
When I depend on the state
I expect the state to be set! (RuntimeError)
./features/step_definitions/steps.rb:6:in `"I depend on the state"'
features/bad_practice_part_2.feature:4:in `I depend on the state'

Feature: Bad practice, part 1

Scenario: Depend on state from a preceding scenario
When I depend on the state
I expect the state to be set! (RuntimeError)
./features/step_definitions/steps.rb:6:in `"I depend on the state"'
features/bad_practice_part_1.feature:7:in `I depend on the state'

Scenario: Set state
Given I set some state

Failing Scenarios:
cucumber features/bad_practice_part_2.feature:3
cucumber features/bad_practice_part_1.feature:6

3 scenarios (2 failed, 1 passed)
3 steps (2 failed, 1 passed)
"""
Loading
Loading