diff --git a/CHANGELOG.md b/CHANGELOG.md index 442b3a7..40bbc4b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,26 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) once a `--set-map-inner-map =:::`. Proven against KubeArmor's `system_monitor.bpf.o` (`kubearmor_visibility`), which then loads across Ubuntu 5.4/5.15, Debian 6.1, Ubuntu 6.8, and AlmaLinux 8 (4.18). +- OCI gadget loading: `--artifact` now accepts an OCI image in addition to a + local `.bpf.o` ELF — a registry reference (e.g. + `ghcr.io/inspektor-gadget/gadget/trace_open:latest`), an OCI layout + directory, or an OCI/docker image archive. bpfcompat extracts the eBPF object + layer (Inspektor Gadget's `application/vnd.gadget.ebpf.program.v1+binary` + media type, with an ELF-magic fallback) and validates it like any other + artifact. This lets gadget authors point the validator straight at a + published gadget. (Requested by Inspektor Gadget maintainer.) +- `--quick`: run the built-in quick-check kernel set (old LTS → recent) instead + of `--matrix`, for a fast local "does it load?" check with no matrix file — + e.g. `bpfcompat test --artifact ghcr.io/org/gadget:tag --quick`. +- Auto-size runtime-sized maps: the validator now gives a default `max_entries` + to maps that ship with `max_entries=0` and whose type requires a positive + size (hash/array/percpu/LRU/stack-trace/LPM/prog-array), matching what the + real loader does at runtime. Types where 0 is meaningful (perf-event-array, + ring/user ringbuf, the `*_STORAGE` local-storage maps) are never touched, and + manifest `max_entries` fixups take precedence. Reported per map in the run + notes. Together with the two items above this makes zero-config gadget + validation work — e.g. Inspektor Gadget's `trace_open` loads with no manifest + (its runtime-sized `ig_build_id` map is auto-sized). - Supply-chain trust signals: GitHub CodeQL static analysis (`.github/workflows/codeql.yml`), OpenSSF Scorecard (`.github/workflows/scorecard.yml`), and Dependabot diff --git a/README.md b/README.md index 0069330..28e5c59 100644 --- a/README.md +++ b/README.md @@ -7,14 +7,23 @@ [![License: Apache-2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE) `bpfcompat` is an open-source compatibility validator for compiled eBPF -artifacts. It runs real libbpf load/attach checks against Linux kernel profiles -and produces JSON/Markdown reports that can fail CI when an artifact regresses. +artifacts. **Test your eBPF across real kernels — locally or in CI.** It boots +real distro kernels in disposable VMs, runs libbpf load/attach checks, and +produces JSON/Markdown reports that can fail CI when an artifact regresses — so +the answer is empirical, not inferred from CO-RE. The core question is simple: > Will this `.bpf.o` load and attach on the kernels I care about, and if not, > what failed? +Point it at a local `.bpf.o` or a **published gadget by OCI reference**, on your +laptop or as a CI gate — `--quick` needs no matrix file: + +```sh +bpfcompat test --artifact ghcr.io/inspektor-gadget/gadget/trace_open:latest --quick +``` + **Live demo:** [bpfcompat.kernelguard.net](https://bpfcompat.kernelguard.net) — upload a `.bpf.o` and see the compatibility matrix. @@ -68,6 +77,23 @@ The red `5.4` row is the point: a kernel below Falco's real floor is flagged `-EINVAL`) and remediation — not a generic "it broke." Reproduce this matrix locally; see [`docs/falco-parity.md`](docs/falco-parity.md). +## Validate a published gadget in one command + +eBPF gadgets ship as OCI images. Point `bpfcompat` at one by reference — it +extracts the eBPF object, auto-sizes runtime-sized maps, and validates it across +kernels with no manifest and no matrix file: + +```sh +bpfcompat test --artifact ghcr.io/inspektor-gadget/gadget/trace_open:latest --quick +``` + +Pulled straight from the registry, Inspektor Gadget's `trace_open`/`trace_exec` +load and attach on 6.1/6.8 and are correctly flagged on 5.4 (the `events` ring +buffer needs ≥ 5.8) — and `trace_open` passes on AlmaLinux 8's backported 4.18, +the textbook "kernel version ≠ feature support" case. Full write-up, including +the `trace_dns` loader-contract finding: +[`docs/case-study-inspektor-gadget.md`](docs/case-study-inspektor-gadget.md). + ## Current Status The project is a serious MVP for compatibility evidence and CI gating. It is @@ -415,6 +441,12 @@ User guide — start here: - [`docs/firecracker-backend.md`](docs/firecracker-backend.md) - [`docs/api-web-ui.md`](docs/api-web-ui.md) +Reference matrices (real, reproducible artifacts): + +- [`docs/case-study-falco-modern-bpf.md`](docs/case-study-falco-modern-bpf.md) — Falco `modern_bpf` across 5 kernels +- [`docs/case-study-enterprise-kernels.md`](docs/case-study-enterprise-kernels.md) — RHEL/Oracle/Amazon/SUSE backported tier +- [`docs/case-study-inspektor-gadget.md`](docs/case-study-inspektor-gadget.md) — published gadgets from OCI, zero config + Internal evidence and program docs (acceptance records, runbooks, and planning notes — useful for contributors, not needed to use the tool): diff --git a/cmd/bpfcompat/main.go b/cmd/bpfcompat/main.go index cd5cd6e..42cf3b1 100644 --- a/cmd/bpfcompat/main.go +++ b/cmd/bpfcompat/main.go @@ -163,13 +163,14 @@ func runTest(args []string) int { fs.SetOutput(os.Stderr) var cfg runner.Config - fs.StringVar(&cfg.ArtifactPath, "artifact", "", "Path to compiled .bpf.o artifact") + fs.StringVar(&cfg.ArtifactPath, "artifact", "", "Compiled .bpf.o artifact: a local ELF file, an OCI gadget (registry ref e.g. ghcr.io/org/gadget:tag), or an OCI image archive/layout") fs.StringVar(&cfg.ArtifactURI, "artifact-uri", "", "Optional remote URI for artifact retrieval metadata (http|https|file)") fs.StringVar(&cfg.ArtifactName, "artifact-name", "", "Logical artifact family name for version history (optional)") fs.StringVar(&cfg.ArtifactVersion, "artifact-version", "", "Artifact version label for version history (optional)") fs.StringVar(&cfg.ArtifactVariant, "artifact-variant", "", "Artifact variant label (optional)") fs.StringVar(&cfg.ValidationMode, "validation-mode", "", "Validation mode: load_only, load_attach, or behavior (default preserves manifest behavior)") fs.StringVar(&cfg.MatrixPath, "matrix", "", "Path to matrix YAML") + fs.BoolVar(&cfg.Quick, "quick", false, "Use the built-in quick-check kernel set instead of --matrix (fast local 'does it load?' check)") fs.StringVar(&cfg.ManifestPath, "manifest", "", "Path to artifact manifest YAML (optional)") fs.StringVar(&cfg.OutPath, "out", "", "Path to JSON report output") fs.StringVar(&cfg.MarkdownPath, "markdown", "", "Path to Markdown report output (optional)") diff --git a/docs/case-study-inspektor-gadget.md b/docs/case-study-inspektor-gadget.md new file mode 100644 index 0000000..07c1fc0 --- /dev/null +++ b/docs/case-study-inspektor-gadget.md @@ -0,0 +1,81 @@ +# Reference matrix: Inspektor Gadget gadgets across kernels (zero-config, from OCI) + +A public, reproducible example of validating real, published eBPF **gadgets** +across kernels — pulled straight from their OCI registry, with no manifest and +no matrix file to write. + +> Independent compatibility test of publicly available artifacts. Not affiliated +> with, sponsored by, or endorsed by Inspektor Gadget or Microsoft. + +## What was tested + +[Inspektor Gadget](https://github.com/inspektor-gadget/inspektor-gadget) ships +its gadgets as OCI images. bpfcompat can pull a gadget by reference, extract the +eBPF object, and validate it across kernels in one command: + +```sh +bpfcompat test --artifact ghcr.io/inspektor-gadget/gadget/trace_open:latest --quick +``` + +- **No manifest, no matrix file.** `--quick` uses a built-in kernel set; the + gadget's runtime-sized maps (compiled `max_entries=0`, sized by IG's loader at + runtime) are auto-sized so the object loads the way the real loader runs it. +- **Validation mode:** load + attach, inside disposable QEMU/KVM VMs running each + exact kernel. + +## Results + +### `trace_open` and `trace_exec` — clean compatibility matrices + +| Kernel | `trace_open` | `trace_exec` | Notes | +|---|---|---|---| +| Ubuntu 20.04 — 5.4 | ❌ fail | ❌ fail | `events` ring buffer requires ≥ 5.8 | +| Debian 12 — 6.1 | ✅ pass (4/4 attach) | ✅ pass (6/6 attach) | runtime-sized `ig_build_id` auto-sized | +| Ubuntu 24.04 — 6.8 | ✅ pass (4/4 attach) | ✅ pass (6/6 attach) | runtime-sized `ig_build_id` auto-sized | + +The `5.4` failure is the point: it is flagged with the exact mechanism (the +`events` ring buffer map cannot be created — ring buffer support lands in 5.8), +not a generic "it broke." Auto-sizing deliberately leaves ring-buffer and +perf-event maps untouched, so the boundary is reported truthfully. + +Run against the full enterprise-aware matrix, `trace_open` additionally **passes +on AlmaLinux 8 (kernel 4.18)** — RHEL backported the ring buffer into 4.18, so +the gadget loads there even though it fails on Ubuntu's *newer* vanilla 5.4. That +is the canonical "kernel version ≠ feature support" case, shown empirically. + +### `trace_dns` — a loader contract, not a kernel limit + +`trace_dns` fails to load on **every** kernel (including 6.8 with BTF), which by +itself signals a loader contract rather than a compatibility boundary. bpfcompat +surfaces the exact reason: + +``` +prog 'ig_trace_dns': missing BPF prog type, check ELF section name 'socket1' +``` + +The DNS gadget is a **socket-filter** program in a `socket1` section — a section +name libbpf cannot map to a program type on its own, so IG's loader sets the type +explicitly. A generic load can't infer it, so the load fails identically across +kernels. This is *not* a kernel-version result; it is exactly the kind of +loader-side detail a gadget's OCI metadata can describe, which is the direction +for deriving load config automatically. + +## Why this matters + +The projects that ship eBPF gadgets (Inspektor Gadget) and the long tail of +third-party gadget authors both need the same answer before shipping: *does this +gadget load and attach on the kernels my users run?* With OCI loading + `--quick` ++ auto-sizing, that answer is a single command against the published artifact — +locally on a laptop or as a CI lane — with the failures classified, not just +counted. + +## Reproduce + +```sh +# any published gadget; --quick needs no matrix file, auto-size needs no manifest +bpfcompat test --artifact ghcr.io/inspektor-gadget/gadget/trace_open:latest --quick +bpfcompat test --artifact ghcr.io/inspektor-gadget/gadget/trace_exec:latest --quick +``` + +Artifacts are pulled from the public registry and validated as shipped; no source +changes. See [docs/quickstart.md](quickstart.md) for the trust model. diff --git a/go.mod b/go.mod index 240cd0d..11d8cb5 100644 --- a/go.mod +++ b/go.mod @@ -3,6 +3,7 @@ module github.com/kernel-guard/bpfcompat go 1.25.0 require ( + github.com/google/go-containerregistry v0.21.7 github.com/prometheus/client_golang v1.23.2 gopkg.in/yaml.v3 v3.0.1 ) @@ -10,12 +11,20 @@ require ( require ( github.com/beorn7/perks v1.0.1 // indirect github.com/cespare/xxhash/v2 v2.3.0 // indirect + github.com/docker/cli v29.5.3+incompatible // indirect + github.com/docker/docker-credential-helpers v0.9.3 // indirect + github.com/klauspost/compress v1.18.6 // indirect github.com/kr/text v0.2.0 // indirect github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect + github.com/opencontainers/go-digest v1.0.0 // indirect + github.com/opencontainers/image-spec v1.1.1 // indirect github.com/prometheus/client_model v0.6.2 // indirect github.com/prometheus/common v0.66.1 // indirect github.com/prometheus/procfs v0.16.1 // indirect + github.com/sirupsen/logrus v1.9.4 // indirect go.yaml.in/yaml/v2 v2.4.2 // indirect - golang.org/x/sys v0.44.0 // indirect + golang.org/x/sync v0.21.0 // indirect + golang.org/x/sys v0.46.0 // indirect google.golang.org/protobuf v1.36.8 // indirect + gotest.tools/v3 v3.5.2 // indirect ) diff --git a/go.sum b/go.sum index fcf6ba7..a0efce1 100644 --- a/go.sum +++ b/go.sum @@ -5,10 +5,16 @@ github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XL github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/docker/cli v29.5.3+incompatible h1:nbEFfz774vBwQ5KRYv7c/AghjReqnGISvrRhzjV0evs= +github.com/docker/cli v29.5.3+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8= +github.com/docker/docker-credential-helpers v0.9.3 h1:gAm/VtF9wgqJMoxzT3Gj5p4AqIjCBS4wrsOh9yRqcz8= +github.com/docker/docker-credential-helpers v0.9.3/go.mod h1:x+4Gbw9aGmChi3qTLZj8Dfn0TD20M/fuWy0E5+WDeCo= github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= -github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo= -github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ= +github.com/google/go-containerregistry v0.21.7 h1:/vPFuVXDjtFREsVArW+0h1CIl5urnOhzei4X2DMW9IU= +github.com/google/go-containerregistry v0.21.7/go.mod h1:kjSbt7/zMsKLWfnHrIvKvhXHUw91jbe9DNjPPJ32gXE= +github.com/klauspost/compress v1.18.6 h1:2jupLlAwFm95+YDR+NwD2MEfFO9d4z4Prjl1XXDjuao= +github.com/klauspost/compress v1.18.6/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ= github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= @@ -17,6 +23,10 @@ github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0 github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= +github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U= +github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM= +github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040= +github.com/opencontainers/image-spec v1.1.1/go.mod h1:qpqAh3Dmcf36wStyyWU+kCeDgrGnAve2nCC8+7h8Q0M= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o= @@ -29,14 +39,18 @@ github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzM github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is= github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ= github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= +github.com/sirupsen/logrus v1.9.4 h1:TsZE7l11zFCLZnZ+teH4Umoq5BhEIfIzfRDZ1Uzql2w= +github.com/sirupsen/logrus v1.9.4/go.mod h1:ftWc9WdOfJ0a92nsE2jF5u5ZwH8Bv2zdeOC42RjbV2g= github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI= go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU= -golang.org/x/sys v0.44.0 h1:ildZl3J4uzeKP07r2F++Op7E9B29JRUy+a27EibtBTQ= -golang.org/x/sys v0.44.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= +golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM= +golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= +golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw= +golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= google.golang.org/protobuf v1.36.8 h1:xHScyCOEuuwZEc6UtSOvPbAT4zRh0xcNRYekJwfqyMc= google.golang.org/protobuf v1.36.8/go.mod h1:fuxRtAxBytpl4zzqUh6/eyUujkJdNiuEkXntxiD/uRU= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= @@ -44,3 +58,5 @@ gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntN gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gotest.tools/v3 v3.5.2 h1:7koQfIKdy+I8UTetycgUqXWSDwpgv193Ka+qRsmBY8Q= +gotest.tools/v3 v3.5.2/go.mod h1:LtdLGcnqToBH83WByAAi/wiwSFCArdFIUV/xxN4pcjA= diff --git a/internal/api/ui.go b/internal/api/ui.go index a9da9b0..1624c8a 100644 --- a/internal/api/ui.go +++ b/internal/api/ui.go @@ -910,7 +910,7 @@ const uiHTML = `
- Technical Preview — CI-first eBPF compatibility gate. Production runtime loading remains disabled in the public demo. + Technical Preview — test your eBPF across real kernels, locally or in CI. Production runtime loading remains disabled in the public demo.