nvx-rs: replace the golden egress-policy corpus with a reference model and property tests #729
Copy link
Copy link
Closed
Feature
Copy link
Description
Goal
Replace tests/fixtures/egress-policy.json with tests whose expectations come from the policy format in doc/run.md: an independent reference model of policies, properties of the rules that they lower to, and a table of the format's grammar. Then make the compiler Rust-native.
Background
- The corpus has 784 policies, each compiled from bytes and from a file: 176 ported test vectors, 200 random ones from seed 359, and 408 written by hand, of which 272 compile. It adds 2 unreadable files and 55 paths printed as
pathlibprints them.every_fixture_case_compiles_as_python_compiled_it,unreadable_files_fail_as_in_python, andpaths_print_as_pathlib_prints_themcheck it, andNVX_EGRESS_POLICY_FIXTUREcan point them at another corpus. The other 42 tests ofsrc/host/network/policy/tests.rsstay. - On Windows, only the corpus reaches 9 lines and 6 branches; what it adds is the breadth of its inputs.
- No policy that compiles uses
NaN,Infinity, duplicate keys, or unpaired surrogates, so strict JSON changes only the messages of policies that are refused already. - OpenVMM's matching doesn't depend on the order of the rules, so their partition and order are free, but must stay deterministic.
Tasks
Replacement tests, in a pull request that doesn't change behavior:
- A reference model in the tests, written from
doc/run.md: whether an action's rules match a packet of a family, address, protocol (TCP, UDP, ICMP, or other), and port; and a parser of the lowered rules,CIDR[:PROTOCOL[:PORT[-LAST]]]. - An exact equivalence check: for each family and protocol, evaluate one point of every cell that the boundaries of the source and lowered networks and port ranges delimit.
- A
proptestgenerator ported from the one with which the retiredscripts/nvx_rs_fixtures.pydrew the seed-359 policies (in36c5bf7af^), which favors overlapping networks, exceptions, and ports at range boundaries, in the MXC and flat forms. - Properties:
allowanddenyeach lower to rules that match exactly what the reference model matches.- No lowered rule is redundant.
- The rules are canonical: sorted, without duplicates or host bits, with no network that holds or merges with another of the same selector, and with no overlapping or adjacent port ranges.
- The order of the rules, duplicated rules, and splitting a network into its halves don't change the lowering.
allowanddenylower independently.- An action is refused exactly when it needs more than 256 rules.
- Compiling bytes, a value, and
NetworkPolicyrules throughopenvmm_argumentsagree. - Compiling is deterministic.
- Arbitrary bytes and mutated policies compile or fail with an error, never panic.
- Grammar tests, one for each rule of the format, that check the field path and the error code (nvx-rs: give CLI errors stable machine-readable codes #724): unknown fields, duplicate keys, a null protocol, IPv6 scopes (Track egress policy compiler bugs: scoped IPv6 validation, range fragmentation #462), host bits, exceptions outside their network or family, ports without
tcp,udp, orany,endPort, the 1 MiB limit, UTF-8, nesting depth, and mixed MXC and flat fields. - Run the properties once over the corpus's 784 inputs before deleting it.
Retirement, in the next pull request:
- Read policies as strict RFC 8259 JSON: refuse
NaN,Infinity, duplicate keys, and nesting deeper than a documented limit. - Messages give the field path and the reason, with Rust's quoting.
- Delete
egress-policy.json, its three tests, andNVX_EGRESS_POLICY_FIXTURE. - Docs: the README's "Egress policy files",
doc/run.md, and theaci_edge_sandboxesREADME where its messages change. Updatescripts/nvx_tools/microvm_tests.pyand the tests ofaci_edge_sandboxeswhere they match messages.
Gate
- The lines and branches that only the corpus reaches, on Windows and Linux (nvx-rs: measure line, branch, and mutation coverage on Linux and Windows #726), are reached by the new tests or deleted.
-
cargo mutantsonsrc/host/network/lowering.rs,src/host/network/policy/mod.rs, andsrc/host/network/address.rsmisses no mutant with the new tests that the corpus catches. - Every policy of the corpus that compiles today compiles to the same rules after the change.
Acceptance criteria
-
egress-policy.jsonis gone, and the properties and grammar tests pass on Linux and Windows. -
proptestis a dev-dependency that builds with the crate's minimum supported Rust version, 1.89.
Metadata
Metadata
Assignees
Labels
No labels
Activity
nvx-rs: let tests lower egress rules beyond OpenVMM's limit
nvx-rs: check egress policy lowering against a reference model
nvx-rs: test each rule of the egress policy grammar
scripts: count every test that replays the egress fixture in its family
nvx-rs: retire the egress policy fixture
nvx-rs: quote values in egress policy messages as Rust does
nvx-rs: name egress policy files as they are given
nvx-rs: quote values in egress policy messages as Rust does
nvx-rs: name egress policy files as they are given
nvx-rs: quote values in egress policy messages as Rust does
nvx-rs: name egress policy files as they are given
nvx-rs: quote values in egress policy messages as Rust does
nvx-rs: name egress policy files as they are given
nvx-rs: quote values in egress policy messages as Rust does
nvx-rs: name egress policy files as they are given
doc: state the testing policy that replaced recorded output
nvx-rs: describe the tests that replaced the golden tests