Add support for 'continue-on-error' in scenario steps and UI compon…

examples/conditional_execution.md

@@ -119,6 +119,46 @@ Access the return value/result data from a specific step. Steps may return data
 
 **Note**: The `result` field accesses the return value/details from the step execution. Use `!= None` to check if a step returned any data.
 
+## Continue on Error
+
+By default, when a step fails the scenario stops. Use `continue-on-error: true` to allow subsequent steps to execute even if the current step fails:
+
+```yaml
+- id: validate_metrics
+  step: Verify Reported Metrics
+  continue-on-error: true  # Scenario continues even if validation fails
+```
+
+This is different from `if: always()` — `continue-on-error` controls what happens **after** the step runs and fails, while `if` controls whether the step runs **at all**.
+
+### Combining with Conditions
+
+`continue-on-error` and `if` can be used together for robust workflows:
+
+```yaml
+steps:
+  - id: run_tests
+    step: Run Tests
+    continue-on-error: true   # Don't halt if tests fail
+
+  - id: collect_results
+    step: Collect Test Results
+    description: Runs even if tests failed above
+
+  - id: cleanup
+    step: Cleanup Resources
+    if: always()               # Runs regardless of any previous failure
+```
+
+### When to Use Each
+
+| Scenario | Use |
+|----------|-----|
+| Step might fail but rest of workflow should continue | `continue-on-error: true` |
+| Step should only run when a previous step failed | `if: failure()` |
+| Step must always run (cleanup, teardown) | `if: always()` |
+| Step should run only on success (default behavior) | No `if` needed, or `if: success()` |
+
 ## Complete Example
 
 ```yaml

examples/workflow_features.md

@@ -21,6 +21,16 @@ Complete guide to advanced workflow features including conditional execution and
 
 **Operators:** `==`, `!=`, `&&`, `||`, `not`
 
+### Continue on Error
+
+```yaml
+- id: step_name
+  step: Operation Name
+  continue-on-error: true  # Scenario continues even if this step fails
+```
+
+By default, a failing step halts the scenario. Use `continue-on-error: true` to allow subsequent steps to execute regardless. This is useful for non-critical checks or when cleanup steps must always run.
+
 ### Loops
 
 ```yaml
@@ -46,6 +56,7 @@ Complete guide to advanced workflow features including conditional execution and
 | **Item Loop** | `loop: { items: [...] }` | Process list of files |
 | **While Loop** | `loop: { while: condition }` | Poll until ready |
 | **Conditional Step** | `if: expression` | Run only on success |
+| **Continue on Error** | `continue-on-error: true` | Don't halt on failure |
 | **Step Status** | `steps.X.status` | Check if step passed |
 | **Step Result** | `steps.X.result` | Access return value |
 | **Loop Index** | `loop.index` | Get iteration number |

scenarios/README.md

@@ -108,6 +108,40 @@ steps:
 - Available status strings: `success`, `failure`, `running`, `skipped`.
 - You can use conditions like `always()`, `success()`, `failure()` and combined references (e.g., `steps.Upload Flight Declaration.status == 'success'`).
 
+### Run Conditions (`if`)
+
+Control when a step executes using the `if` field:
+
+```yaml
+steps:
+  - step: Cleanup Resources
+    if: always()    # Always runs, even after failures
+
+  - step: Deploy
+    if: success()   # Runs only if all previous steps succeeded
+
+  - step: Rollback
+    if: failure()   # Runs only if a previous step failed
+
+  - step: Custom Check
+    if: steps.test.status == 'success'  # Check a specific step
+```
+
+### Continue on Error (`continue-on-error`)
+
+By default, a failing step halts the scenario. Set `continue-on-error: true` to allow subsequent steps to execute even if this step fails:
+
+```yaml
+steps:
+  - step: Run Flaky Test
+    continue-on-error: true   # Scenario continues even if this fails
+
+  - step: Cleanup Resources
+    if: always()
+```
+
+This is useful for non-critical validation steps or cleanup flows where you want the scenario to keep going.
+
 ## Validation
 To validate scenarios locally:
 

scenarios/flight-declarations/add_flight_declaration.yaml

@@ -14,3 +14,4 @@ steps:
   arguments:
     state: ENDED
 - step: Teardown Flight Declaration
+  if: always()

scenarios/flight-declarations/add_flight_declaration_via_operational_intent.yaml

@@ -14,3 +14,4 @@ steps:
   arguments:
     state: ENDED
 - step: Teardown Flight Declaration
+  if: always()

scenarios/flight-declarations/bulk_add_flight_declarations.yaml

@@ -4,9 +4,11 @@ steps:
 - step: Setup Two Flight Declarations
 - id: teardown_first_declaration
   step: Teardown Flight Declaration
+  if: always()
   arguments:
     flight_declaration_id: ${{ steps.Setup Two Flight Declarations.result.declarations[0].id }}
 - id: teardown_second_declaration
   step: Teardown Flight Declaration
+  if: always()
   arguments:
     flight_declaration_id: ${{ steps.Setup Two Flight Declarations.result.declarations[1].id }}

scenarios/flight-declarations/bulk_add_flight_declarations_via_operational_intents.yaml

@@ -4,9 +4,11 @@ steps:
 - step: Setup Two Operational Intents
 - id: teardown_first_declaration
   step: Teardown Flight Declaration
+  if: always()
   arguments:
     flight_declaration_id: ${{ steps.Setup Two Operational Intents.result.declarations[0].id }}
 - id: teardown_second_declaration
   step: Teardown Flight Declaration
+  if: always()
   arguments:
     flight_declaration_id: ${{ steps.Setup Two Operational Intents.result.declarations[1].id }}

scenarios/sdsp-f3623/sdsp_heartbeat.yaml

@@ -29,3 +29,4 @@ steps:
     arguments:
       action: STOP
       surveillance_session_id: ${{ steps.Generate UUID.result }}
+    if: always()

scenarios/sdsp-f3623/sdsp_track.yaml

@@ -33,3 +33,4 @@ steps:
     action: STOP
   needs:
   - stream_air_traffic
+  if: always()

scenarios/sdsp-f3623/verify_sdsp_metrics.yaml

@@ -27,6 +27,7 @@ steps:
 - step: Verify Reported Metrics in Flight Blender
   needs:
     - stream_air_traffic
+  continue-on-error: true
   arguments:
     observations: ${{ steps.stream_air_traffic.result.observations }}
     # session_id: ${{ steps.generated_sdsp_session_id.result }}
@@ -37,3 +38,4 @@ steps:
     action: STOP
   needs:
   - stream_air_traffic
+  if: always()

scenarios/standard-scenarios/F1_flow_no_telemetry_with_user_input.yaml

@@ -16,3 +16,4 @@ steps:
   arguments:
     state: ENDED
 - step: Teardown Flight Declaration
+  if: always()

scenarios/standard-scenarios/F1_happy_path.yaml

@@ -22,4 +22,5 @@ steps:
     state: ENDED
   description: Marks the operation as ended after the flight is complete.
 - step: Teardown Flight Declaration
+  if: always()
   description: Cleans up the flight declaration and any associated resources.

scenarios/standard-scenarios/F2_contingent_path.yaml

@@ -18,3 +18,4 @@ steps:
   arguments:
     state: ENDED
 - step: Teardown Flight Declaration
+  if: always()

scenarios/standard-scenarios/F3_non_conforming_path.yaml

@@ -29,4 +29,5 @@ steps:
   arguments:
     state: ENDED
 - step: Teardown Flight Declaration
+  if: always()
   description: Cleanup.

scenarios/standard-scenarios/F5_non_conforming_path.yaml

@@ -21,3 +21,4 @@ steps:
   arguments:
     state: ENDED
 - step: Teardown Flight Declaration
+  if: always()

src/openutm_verification/core/execution/conditions.py

@@ -32,9 +32,9 @@ def __init__(self, steps: dict[str, StepResult[Any]], loop_context: dict[str, An
         self.loop_context = loop_context or {}
         self.last_step_status: Status | None = None
 
-        # Determine last step status (excluding skipped steps)
+        # Determine last step status (excluding skipped steps and steps with continue_on_error that failed)
         if steps:
-            completed_steps = [s for s in steps.values() if s.status != Status.SKIP]
+            completed_steps = [s for s in steps.values() if s.status != Status.SKIP and not (s.continue_on_error and s.status == Status.FAIL)]
             if completed_steps:
                 self.last_step_status = completed_steps[-1].status
 

src/openutm_verification/core/execution/definitions.py

@@ -24,6 +24,11 @@ class StepDefinition(BaseModel):
     description: str | None = None
     if_condition: str | None = Field(default=None, alias="if", description="Conditional expression to determine if step should run")
     loop: LoopConfig | None = Field(default=None, description="Loop configuration for repeating the step")
+    continue_on_error: bool = Field(
+        default=False,
+        alias="continue-on-error",
+        description="If true, subsequent steps will still execute even if this step fails. Default is false (stop on failure).",
+    )
 
 
 class GroupDefinition(BaseModel):

src/openutm_verification/core/reporting/reporting_models.py

@@ -44,6 +44,7 @@ class StepResult(BaseModel, Generic[T]):
     result: T = None  # type: ignore
     error_message: str | None = None
     logs: list[str] = []
+    continue_on_error: bool = False
 
 
 class ScenarioResult(BaseModel):

src/openutm_verification/server/router.py

@@ -105,7 +105,7 @@ async def get_scenario(scenario: str):
     """Get the content of a specific scenario."""
     try:
         scenario_def = load_yaml_scenario_definition(scenario)
-        return scenario_def.model_dump()
+        return scenario_def.model_dump(by_alias=True)
     except FileNotFoundError:
         raise HTTPException(status_code=404, detail="Scenario not found")
     except Exception as e:
@@ -131,7 +131,7 @@ async def save_scenario(name: str, scenario: ScenarioDefinition):
 
     try:
         # Convert Pydantic model to dict, excluding None values to keep YAML clean
-        data = scenario.model_dump(exclude_none=True, exclude_defaults=True)
+        data = scenario.model_dump(by_alias=True, exclude_none=True, exclude_defaults=True)
 
         with open(file_path, "w") as f:
             yaml.dump(data, f, sort_keys=False, default_flow_style=False)

src/openutm_verification/server/runner.py

@@ -626,10 +626,17 @@ async def _execute_group(
 
             # If step failed and it's not allowed to fail, break the group
             if result.status == Status.FAIL:
-                logger.error(f"Group step '{group_step.id}' failed, stopping group execution")
-                # Mark remaining steps as SKIP
-                self._mark_remaining_group_steps_skipped(group, group_name, loop_context, executed_step_indices)
-                break
+                if group_step.continue_on_error:
+                    logger.warning(f"Group step '{group_step.id}' failed, but continue-on-error=true — continuing group")
+                    result.continue_on_error = True
+                    if self.session_context:
+                        with self.session_context:
+                            self.session_context.update_result(result)
+                else:
+                    logger.error(f"Group step '{group_step.id}' failed, stopping group execution")
+                    # Mark remaining steps as SKIP
+                    self._mark_remaining_group_steps_skipped(group, group_name, loop_context, executed_step_indices)
+                    break
 
         return results
 
@@ -663,6 +670,15 @@ def _mark_remaining_group_steps_skipped(
             with self.session_context:
                 self.session_context.update_result(skipped_result)
 
+    def _mark_results_continue_on_error(self, results: List[StepResult]) -> None:
+        """Mark failed results as continue_on_error and update them in session context."""
+        for result in results:
+            if result.status == Status.FAIL:
+                result.continue_on_error = True
+                if self.session_context:
+                    with self.session_context:
+                        self.session_context.update_result(result)
+
     async def _wait_for_dependencies(self, step: StepDefinition) -> None:
         """Wait for any declared dependencies (by step ID) before executing a step."""
         if not step.needs:
@@ -779,30 +795,50 @@ async def run_scenario(self, scenario: ScenarioDefinition) -> List[StepResult]:
             # Wait for declared dependencies (useful for background steps)
             await self._wait_for_dependencies(step)
 
+            should_continue_on_error = step.continue_on_error
+
             # Check if this step references a group
             if self._is_group_reference(step.step, scenario):
                 # Handle loop execution for groups
                 if step.loop:
                     loop_results = await self._execute_loop_for_group(step, scenario)
                     if loop_results and any(r.status == Status.FAIL for r in loop_results):
-                        logger.error(f"Loop for group '{step.id}' failed, breaking scenario")
-                        break
+                        if should_continue_on_error:
+                            logger.warning(f"Loop for group '{step.id}' failed, but continue-on-error=true — continuing")
+                            self._mark_results_continue_on_error(loop_results)
+                        else:
+                            logger.error(f"Loop for group '{step.id}' failed, breaking scenario")
+                            break
                 else:
                     group_results = await self._execute_group(step, scenario)
                     if any(r.status == Status.FAIL for r in group_results):
-                        logger.error(f"Group '{step.id}' failed, breaking scenario")
-                        break
+                        if should_continue_on_error:
+                            logger.warning(f"Group '{step.id}' failed, but continue-on-error=true — continuing")
+                            self._mark_results_continue_on_error(group_results)
+                        else:
+                            logger.error(f"Group '{step.id}' failed, breaking scenario")
+                            break
             else:
                 # Regular step execution
                 # Handle loop execution
                 if step.loop:
                     loop_results = await self._execute_loop(step)
                     # Check if any loop iteration failed
                     if loop_results and any(r.status == Status.FAIL for r in loop_results):
-                        logger.error(f"Loop for step '{step.id}' failed, breaking scenario")
-                        break
+                        if should_continue_on_error:
+                            logger.warning(f"Loop for step '{step.id}' failed, but continue-on-error=true — continuing")
+                            self._mark_results_continue_on_error(loop_results)
+                        else:
+                            logger.error(f"Loop for step '{step.id}' failed, breaking scenario")
+                            break
                 else:
-                    await self.execute_single_step(step)
+                    result = await self.execute_single_step(step)
+                    if result.status == Status.FAIL and should_continue_on_error:
+                        logger.warning(f"Step '{step.id}' failed, but continue-on-error=true — continuing")
+                        result.continue_on_error = True
+                        if self.session_context:
+                            with self.session_context:
+                                self.session_context.update_result(result)
         # An empty scenario (or one whose every step was skipped before any
         # `with self.session_context:` block ran) leaves session_context.state
         # unset; return an empty list rather than crashing.

tests/test_conditions.py

@@ -171,3 +171,42 @@ def test_loop_index_boundary_conditions(self):
         evaluator = ConditionEvaluator(steps, loop_context)
         assert evaluator.evaluate("loop.index < 100") is True
         assert evaluator.evaluate("loop.index >= 99") is True
+
+    def test_success_ignores_continue_on_error_steps(self):
+        """Test success() ignores failed steps with continue_on_error=True."""
+        steps = {
+            "step1": StepResult(id="step1", name="Step 1", status=Status.PASS, duration=1.0),
+            "step2": StepResult(id="step2", name="Step 2", status=Status.FAIL, duration=1.0, continue_on_error=True),
+        }
+        evaluator = ConditionEvaluator(steps)
+        # step2 failed but has continue_on_error, so success() should still be True
+        assert evaluator.evaluate("success()") is True
+
+    def test_failure_ignores_continue_on_error_steps(self):
+        """Test failure() ignores failed steps with continue_on_error=True."""
+        steps = {
+            "step1": StepResult(id="step1", name="Step 1", status=Status.PASS, duration=1.0),
+            "step2": StepResult(id="step2", name="Step 2", status=Status.FAIL, duration=1.0, continue_on_error=True),
+        }
+        evaluator = ConditionEvaluator(steps)
+        # step2 failed but has continue_on_error, so last_step_status is PASS (from step1)
+        assert evaluator.evaluate("failure()") is False
+
+    def test_continue_on_error_step_status_still_queryable(self):
+        """Test that individual step status is still queryable even with continue_on_error."""
+        steps = {
+            "step1": StepResult(id="step1", name="Step 1", status=Status.PASS, duration=1.0),
+            "step2": StepResult(id="step2", name="Step 2", status=Status.FAIL, duration=1.0, continue_on_error=True),
+        }
+        evaluator = ConditionEvaluator(steps)
+        # Direct step status queries should still reflect the actual status
+        assert evaluator.evaluate("steps.step2.status == 'failure'") is True
+
+    def test_success_fails_with_non_continue_on_error_step(self):
+        """Test success() still returns False for failed steps without continue_on_error."""
+        steps = {
+            "step1": StepResult(id="step1", name="Step 1", status=Status.PASS, duration=1.0),
+            "step2": StepResult(id="step2", name="Step 2", status=Status.FAIL, duration=1.0, continue_on_error=False),
+        }
+        evaluator = ConditionEvaluator(steps)
+        assert evaluator.evaluate("success()") is False