feat: TC8 SOME/IP conformance test suite for someipd by jorgecasal · Pull Request #60 · eclipse-score/inc_someip_gateway

jorgecasal · 2026-03-30T13:13:43Z

Summary

Adds a TC8 SOME/IP conformance test suite for someipd, covering the OA TC8 ECU Test Specification. The tests send and receive raw SOME/IP packets over loopback sockets, no gatewayd needed. A new --tc8-standalone flag in someipd lets it offer services directly without the full gateway stack.

Scope: 183 test cases across 10 test modules, plus documentation and CI integration.

What's included

Tests (tests/tc8_conformance/)

Service Discovery: offer/find lifecycle, timing, reboot, multicast, format compliance, robustness against malformed packets
SOME/IP Message Format: header fields, request/response, fire-and-forget, error handling
Event Notification: subscribe, initial event, cyclic events, multicast delivery
Field Conformance: get/set, initial value on subscribe
Multi-service: multiple services in a single vsomeip instance
3 tests are marked xfail for known vsomeip 3.6.1 limitations

Documentation (docs/)

docs/architecture/tc8_conformance_testing.rst — standalone mode design and test structure
docs/tc8_conformance/requirements.rst — requirements with AUTOSAR PRS traceability IDs
docs/tc8_conformance/test_specification.rst — maps each requirement to test cases
docs/tc8_conformance/traceability.rst — requirement-to-test traceability matrix

someipd changes (src/someipd/main.cpp)

--tc8-standalone flag: skips gatewayd IPC, offers test services directly
Uses ET_FIELD for field events so vsomeip sends the initial value on subscribe

CI (.github/workflows/build_and_test_host.yml)

New step runs TC8 tests with --config=tc8 (creates a network namespace per test, no sudo needed)

Build/config changes

.bazelrc: --config=tc8 opt-in configuration; TC8 tests excluded from bazel test //... by default
json_schema_validator upgraded to 2.4.0; removed unused jsonschema Python dep
tests/integration/BUILD.bazel: migrated from py_pytest to score_py_pytest
.pre-commit-config.yaml: raised large-file limit from 50KB to 125KB for TC8 test files

Test plan

bazel test --config=tc8 //tests/tc8_conformance/... passes locally
bazel build //... and bazel test //... pass (TC8 excluded by default)
bazel run //:copyright.check passes
bazel build //docs/... builds without errors
CI workflow green on this branch

Wire-level pytest tests verifying someipd against the OA TC8 ECU Test Specification v3.0 Chapter 5: service discovery (phases, reboot, timing), message format, event notification, field lifecycle, and TCP transport. - someipd --tc8-standalone mode: request/response, field GET/SET, UDP and TCP events; offer_event uses ET_FIELD so vsomeip delivers the cached field value to new subscribers immediately on subscribe (is_field in JSON config is not honoured in programmatic offer_event calls) - vsomeip config templates for SD timing and service/event/field/TCP tests, with JSON Schema validation at lint time - Architecture doc, requirements, test specification and OA spec traceability - Bazel port isolation via the env attribute: each target receives unique SD and service ports enabling medium targets to run in parallel; timing-sensitive and reboot lifecycle targets retain the exclusive tag for serial execution

github-actions · 2026-03-30T13:17:40Z

License Check Results

🚀 The license check job ran with the Bazel command:

bazel run //:license-check

Status: ⚠️ Needs Review

Click to expand output

[License Check Output]
Extracting Bazel installation...
Starting local Bazel server (8.4.1) and connecting to it...
INFO: Invocation ID: 63d04ee3-d435-421e-9d40-f5506d725982
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Loading: 
Loading: 4 packages loaded
Loading: 4 packages loaded
    currently loading: 
Loading: 4 packages loaded
    currently loading: 
Analyzing: target //:license-check (5 packages loaded, 0 targets configured)
Analyzing: target //:license-check (5 packages loaded, 0 targets configured)

Analyzing: target //:license-check (44 packages loaded, 10 targets configured)

Analyzing: target //:license-check (91 packages loaded, 10 targets configured)

Analyzing: target //:license-check (91 packages loaded, 10 targets configured)

Analyzing: target //:license-check (146 packages loaded, 2771 targets configured)

Analyzing: target //:license-check (161 packages loaded, 5566 targets configured)

Analyzing: target //:license-check (169 packages loaded, 8311 targets configured)

Analyzing: target //:license-check (169 packages loaded, 8311 targets configured)

Analyzing: target //:license-check (174 packages loaded, 10442 targets configured)

Analyzing: target //:license-check (174 packages loaded, 10442 targets configured)

Analyzing: target //:license-check (174 packages loaded, 10442 targets configured)

INFO: Analyzed target //:license-check (175 packages loaded, 12021 targets configured).
[9 / 17] [Prepa] Creating source manifest for @@score_tooling+//dash/tool/formatters:dash_format_converter [for tool] ... (2 actions, 0 running)
INFO: From Generating Dash formatted dependency file ...:
INFO: Successfully converted 2 packages from Cargo.lock to bazel-out/k8-fastbuild/bin/formatted.txt
INFO: Found 1 target...
Target //:license.check.license_check up-to-date:
  bazel-bin/license.check.license_check
  bazel-bin/license.check.license_check.jar
INFO: Elapsed time: 209.867s, Critical Path: 0.28s
INFO: 17 processes: 5 disk cache hit, 12 internal.
INFO: Build completed successfully, 17 total actions
INFO: Running command line: bazel-bin/license.check.license_check ./formatted.txt <args omitted>
usage: org.eclipse.dash.licenses.cli.Main [-batch <int>] [-cd <url>]
       [-confidence <int>] [-ef <url>] [-excludeSources <sources>] [-help] [-lic
       <url>] [-project <shortname>] [-repo <url>] [-review] [-summary <file>]
       [-timeout <seconds>] [-token <token>]

github-actions · 2026-03-30T13:23:01Z

The created documentation from the pull request is available at: docu-html

- Raise check-added-large-files limit from 50 KB to 125 KB to accommodate large but legitimate TC8 test modules and the RST test specification document - Add REUSE.toml annotation for tests/tc8_conformance/config/*.json so the four vsomeip JSON config files pass reuse-lint-file - Prepend Apache-2.0 SPDX comment headers to the two TC8 README.md files that were missing copyright notices

TC8 conformance tests require special setup (multicast route, env vars) and already run in a dedicated step; avoid double-execution via -tc8 filter.

mikehaller · 2026-03-30T14:35:28Z

Documentation overall looks very good, reviewed the current state at https://eclipse-score.github.io/inc_someip_gateway/pr-60/architecture/tc8_conformance_testing.html - not in detail, but general sections and impression. Nice!

On my local checkout, running the tests works with the expected failures:

Executed 10 out of 10 tests: 10 fail locally
3 failed, 2 skipped in 59.50s

$ time bazel test --test_tag_filters=tc8 --test_env=TC8_HOST_IP=127.0.0.1 //tests/tc8_conformance/...

Result:

FAILED ../score_tooling+/python_basics/score_pytest::TestSDClientStopSubscribe::test_ets_084_stop_subscribe_ceases_events - AssertionError: ETS_084: Prerequisite failed — no SubscribeAck received
FAILED ../score_tooling+/python_basics/score_pytest::TestSDClientReboot::test_ets_081_reboot_flag_set_after_first_restart - AssertionError: ETS_081: No SD messages captured after restart
FAILED ../score_tooling+/python_basics/score_pytest::TestSDClientReboot::test_ets_082_reboot_flag_set_after_second_restart - AssertionError: ETS_082: No SD messages captured after second restart

real 6m16.073s

mikehaller · 2026-03-30T15:43:34Z

I missed setting up the multicast... now the tests run fine locally as well, can confirm.

# Set up the multicast route locally
$ sudo ip route add 224.0.0.0/4 dev lo

# Re-run the tests
$ time bazel test --test_tag_filters=tc8 --test_env=TC8_HOST_IP=127.0.0.1 //tests/tc8_conformance/...
INFO: Analyzed 13 targets (0 packages loaded, 0 targets configured).
INFO: Found 3 targets and 10 test targets...
INFO: Elapsed time: 214.786s, Critical Path: 182.28s
INFO: 11 processes: 2 action cache hit, 1 internal, 10 linux-sandbox.
INFO: Build completed successfully, 11 total actions
//tests/tc8_conformance:tc8_event_notification                           PASSED in 24.6s
//tests/tc8_conformance:tc8_field_conformance                            PASSED in 3.8s
//tests/tc8_conformance:tc8_message_format                               PASSED in 42.4s
//tests/tc8_conformance:tc8_multi_service                                PASSED in 10.0s
//tests/tc8_conformance:tc8_sd_client                                    PASSED in 19.0s
//tests/tc8_conformance:tc8_sd_format                                    PASSED in 49.3s
//tests/tc8_conformance:tc8_sd_phases_timing                             PASSED in 7.2s
//tests/tc8_conformance:tc8_sd_reboot                                    PASSED in 4.1s
//tests/tc8_conformance:tc8_sd_robustness                                PASSED in 5.6s
//tests/tc8_conformance:tc8_service_discovery                            PASSED in 181.6s

Executed 10 out of 10 tests: 10 tests pass.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.

real    3m34.876s
user    0m0.203s
sys     0m0.305s

mikehaller

As the TC8 Conformity Tests is a complete new part for someip gateway, I'll approve the current state of the PR. It's a very good step forward.

There is one issue regarding the main.cpp where I didn't fully grasp the plans and hence would accept it if it's just temporary.

- Use local json_schema_validator rule instead of @score_communication - Add integrity hashes to download_archive deps for reproducible builds - Add component descriptions for gatewayd and someipd in architecture docs

Use `unshare --user --net` to create a private network namespace per test, eliminating the need for `sudo ip route add` before running TC8 conformance tests. Add `--config=tc8` Bazel config that sets the tag filter, host IP, and --run_under wrapper automatically. Also exclude TC8 from `bazel test //...` via default tag filter and standardize terminology from "real NIC" to "non-loopback interface".

- Add shebang and is_executable = True to json_schema_validator.bzl script output. - Simplify TC8 network namespace wrapper documentation across .bazelrc, RST docs, README.md, and the wrapper script comments - Add a warning on namespace fallback in tc8_net_wrapper.sh

…ance_test_infrastructure

Simplify the PlantUML hierarchy diagram by removing documentation- artifact nodes (test_specification.rst, traceability.rst) that were mistaken for additional requirement levels. Replace hardcoded /tmp/vsomeip-tc8.log with __TC8_LOG_DIR__ placeholder in all TC8 config templates, substituted at render time with pytest tmp_path.

…ance_test_infrastructure

…ation

…tion

…ance_test_infrastructure # Conflicts: # MODULE.bazel

…ance_test_infrastructure # Conflicts: # docs/architecture/components.rst

…ance_test_infrastructure

…lity gaps

…o msg_malformed

…mance docs

…ceability

jorgecasal requested a review from mikehaller March 30, 2026 13:14

jorgecasal changed the title ~~Add TC8 SOME/IP conformance test suite for someipd~~ RFC: TC8 SOME/IP conformance test suite for someipd Mar 30, 2026

jorgecasal added documentation Improvements or additions to documentation github_actions Pull requests that update GitHub Actions code python Pull requests that update python code labels Mar 30, 2026

jorgecasal added 2 commits March 30, 2026 15:28

Exclude TC8 tests from general test sweep

a0e345b

TC8 conformance tests require special setup (multicast route, env vars) and already run in a dedicated step; avoid double-execution via -tc8 filter.

jorgecasal added the enhancement New feature or request label Mar 30, 2026

jorgecasal force-pushed the jorgecasal_tc8_conformance_test_infrastructure branch from c507e28 to 63d628d Compare March 30, 2026 14:16

Mark known vsomeip 3.6.1 TC8 failures as xfail to unblock CI.

8420326

jorgecasal force-pushed the jorgecasal_tc8_conformance_test_infrastructure branch from 63d628d to 8420326 Compare March 30, 2026 14:17

jorgecasal added 2 commits March 30, 2026 16:26

Update workflow step name for TC8 conformance tests.

82388be

Fix small inconsistency in docs/tc8_conformance/index.rst

7639458

jorgecasal requested review from NEOatNHNG, lurtz and mariuswbr March 30, 2026 14:36

jorgecasal commented Mar 30, 2026

View reviewed changes

Comment thread tests/tc8_conformance/BUILD.bazel Outdated

Update CI/CD Integration docs to reflect current workflow

a549727

mikehaller approved these changes Mar 30, 2026

View reviewed changes

Comment thread tests/tc8_conformance/README.md Outdated

Comment thread src/someipd/main.cpp

jorgecasal changed the title ~~RFC: TC8 SOME/IP conformance test suite for someipd~~ TC8 SOME/IP conformance test suite for someipd Mar 31, 2026

jorgecasal commented Mar 31, 2026

View reviewed changes

Comment thread MODULE.bazel Outdated

jorgecasal added 2 commits March 31, 2026 07:39

Clarify SOME/IP server/client terminology in TC8 test spec.

146c350

Update known limitations table to reflect xfail markers.

9f84116

jorgecasal self-assigned this Mar 31, 2026

Align TC8 conformance with upstream main after merge

2dfcff0

- Use local json_schema_validator rule instead of @score_communication - Add integrity hashes to download_archive deps for reproducible builds - Add component descriptions for gatewayd and someipd in architecture docs

jorgecasal added 2 commits April 10, 2026 08:55

TC8 tests: skip gracefully when environment is not configured.

924cbc8

lurtz reviewed Apr 10, 2026

View reviewed changes

Comment thread tests/tc8_conformance/tc8_net_wrapper.sh Outdated

Comment thread tests/tc8_conformance/tc8_net_wrapper.sh Outdated

jorgecasal force-pushed the jorgecasal_tc8_conformance_test_infrastructure branch from 0fae264 to 622244e Compare April 10, 2026 15:00

jorgecasal force-pushed the jorgecasal_tc8_conformance_test_infrastructure branch from 622244e to cf3a5d8 Compare April 10, 2026 15:01

docs: trim TC8 architecture doc and move coverage to traceability.

70b305e

jorgecasal force-pushed the jorgecasal_tc8_conformance_test_infrastructure branch from e3cf506 to 70b305e Compare April 13, 2026 07:03

lurtz mentioned this pull request Apr 15, 2026

fix: Update bazel cache on "merge_group" event eclipse-score/cicd-workflows#97

Merged

jorgecasal added 3 commits April 20, 2026 07:37

Merge remote-tracking branch 'score/main' into jorgecasal_tc8_conform…

87b8da1

…ance_test_infrastructure

Merge remote-tracking branch 'score/main' into jorgecasal_tc8_conform…

7e5dc82

…ance_test_infrastructure

jorgecasal changed the title ~~TC8 SOME/IP conformance test suite for someipd~~ feat: TC8 SOME/IP conformance test suite for someipd May 5, 2026

jorgecasal added 5 commits May 6, 2026 09:33

fix(tc8): use class-scoped DUT fixture to prevent SD multicast degrad…

a48d567

…ation

refactor(tc8): use linux-sandbox instead of unshare for network isola…

7b7d20a

…tion

fix(tc8): add requires-fakeroot tag for linux-sandbox network setup

9b9dbe7

fix(tc8): add unshare fallback when linux-sandbox namespace unavailable

d72d5c9

fix(tc8): enforce linux-sandbox strategy for network namespace isolation

bca3b5b

jorgecasal force-pushed the jorgecasal_tc8_conformance_test_infrastructure branch from fc1900a to bca3b5b Compare May 6, 2026 13:59

fix(tc8): improve test robustness for slow CI/container environments

283f0f5

jorgecasal removed request for NEOatNHNG and mariuswbr May 6, 2026 15:26

jorgecasal added 4 commits May 22, 2026 09:07

Merge remote-tracking branch 'score/main' into jorgecasal_tc8_conform…

9effb57

…ance_test_infrastructure # Conflicts: # MODULE.bazel

Merge remote-tracking branch 'score/main' into jorgecasal_tc8_conform…

ec75da5

…ance_test_infrastructure # Conflicts: # docs/architecture/components.rst

Merge remote-tracking branch 'score/main' into jorgecasal_tc8_conform…

1a29aad

…ance_test_infrastructure

fix: Migrate someipd logging to score::mw::log and close TC8 traceabi…

47e56ae

…lity gaps

jorgecasal force-pushed the jorgecasal_tc8_conformance_test_infrastructure branch from adba8e1 to 47e56ae Compare June 24, 2026 08:01

jorgecasal added 3 commits June 24, 2026 10:51

fix: TC8-MSG-031 requirement label corrected msg_malformed_handling t…

6d126c8

…o msg_malformed

docs: remove redundant S-CORE guidelines explanations from TC8 confor…

2c91c23

…mance docs

docs: remove redundant coverage tables and example lists from TC8 tra…

5c7f66f

…ceability

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: TC8 SOME/IP conformance test suite for someipd#60

feat: TC8 SOME/IP conformance test suite for someipd#60
jorgecasal wants to merge 33 commits into
eclipse-score:mainfrom
etas-contrib:jorgecasal_tc8_conformance_test_infrastructure

jorgecasal commented Mar 30, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Mar 30, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Mar 30, 2026

Uh oh!

mikehaller commented Mar 30, 2026

Uh oh!

Uh oh!

mikehaller commented Mar 30, 2026

Uh oh!

mikehaller left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

jorgecasal commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What's included

Test plan

Uh oh!

github-actions Bot commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

License Check Results

Uh oh!

github-actions Bot commented Mar 30, 2026

Uh oh!

mikehaller commented Mar 30, 2026

Uh oh!

Uh oh!

mikehaller commented Mar 30, 2026

Uh oh!

mikehaller left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jorgecasal commented Mar 30, 2026 •

edited

Loading

github-actions Bot commented Mar 30, 2026 •

edited

Loading