Improve error reporting

[PaulN2112CGE]

Goal: Transform technical failures into clear, actionable user guidance while maintaining full diagnostic capability for development teams.

Requirements

User-Facing Error Reporting

• Display user-friendly error messages to end users
• Map errors as close as possible to the actual user input location
• Categorize errors clearly:
  • Software bugs → "Please contact support"
  • Transformation errors → Explain what failed during processing
  • Validation errors → Highlight input issues with corrective guidance

Comprehensive Feedback System

• Support multiple severity levels: Info, Warnings, Errors
• Allow successful actions to report warnings/info alongside completion
• Provide actionable feedback for partial successes

Error State Preservation

• Maintain application state at the point of error
• Preserve user input and context for detailed analysis
• Enable users to correct and retry without losing work

Traceability & Debugging

• Log sufficient context for support team investigation

[PaulN2112CGE]

Design

Current Major Design Issues

1. Errors are reported with cryptic error messages
2. Warning cannot be presented at all
3. Visual feedback in the diagram is only provided for validations performed in the BPMNS4 editor client
4. When tests are generated, a (too) detailed error report is sent to the user, but the actual intermediate state up to where the error occurred is cleared
5. Validations of the p-spec cannot be mapped to BPMN elements

Solution

Extend P-Spec Validation

1. The p-spec will become the main validation point. When a p-spec is valid, downstream transformations should succeed as well. Which validations need to be added remains open at this point.
2. P-spec validation can be traced back to BPMN elements. As a consequence, the user receives feedback directly in the diagram (including warnings).

Retain Intermediate State

1. When test generation (or other transformations) completes, a dialog is always displayed showing errors, warnings, and info messages, regardless of success or failure. The dialog provides a download link to retrieve the generated tests. The download link keeps the dialog open, allowing users to review messages and download multiple times if needed. In case of failures, the download includes all artifacts up to the point where the error occurred.
2. The report of the test generation and simulation is included in the downloaded zip file.
3. If simulation cannot be started, it is treated similarly to test generation failures: the same dialog is displayed with errors, warnings, and info messages, and a download link is provided to retrieve all available artifacts up to the point of failure. This allows specialists to further analyze the root cause if needed.

[raulmonti]

Hi @PaulN2112CGE, this looks very nice. It got me thinking that I usually get the feedback that the zip file with the tests contains too much besides the actual test. Of course this is not an important complain IMO, but maybe we could think about re-structuring it, so that there are maybe 2 folders at the top level: the TESTS folder and the OTHER-THIGNS folder? Kind of the user folder and the specialists folder, referring to the roles you use above.