feat(zone-tiny): add minimal kernel flavor

[89luca89]

zone-tiny is an x86_64 zone kernel built from tinyconfig plus an explicit allowlist, a strict subset of zone. verify-config.py gates a required/forbidden/stickiness contract before the compile, so upstream defconfig churn cannot
leak in.

Validated against protect's KVM and Xen e2e suites:

zone-tiny drops all 55 loadable modules and ~730 enabled symbols (2072 -> 1343), shrinking the OCI payload 45% (20.6 -> 11.2 MB), almost all of it addons collapsing 6.8 -> 0.05 MB.
It cuts boot time ~27% under Cloud Hypervisor: 288.8 ms vs 398.5 ms mean over 100 runs.

This tinyconf is for both xen and kvm, but it's not carrying any driver in case of pass through, we should come up with a specific flavour for that (or just use the regular zone kernel).

  ┌───────────┬─────────────┬───────────┬───────────────────────┬───────────────┐
  │  flavor   │ built-in =y │ module =m │ disabled (is not set) │ enabled (y+m) │
  ├───────────┼─────────────┼───────────┼───────────────────────┼───────────────┤
  │ zone      │ 2017        │ 55        │ 1900                  │ 2072          │
  ├───────────┼─────────────┼───────────┼───────────────────────┼───────────────┤
  │ zone-tiny │ 1343        │ 0         │ 1376                  │ 1343          │
  └───────────┴─────────────┴───────────┴───────────────────────┴───────────────┘

default_distrobox:linux-kernel-oci$ ./hack/bench/compare-kernel-size.sh && ./hack/bench/measure-boot-time.sh

FLAVOR           KERNEL     ADDONS    CONFIG      META   TOTAL(MB)
---------------------------------------------------------------------
zone             13.782      6.797     0.029     0.000      20.608
zone-tiny        11.126      0.051     0.022     0.000      11.199

delta zone -> zone-tiny:  9.409 MB smaller  (45%)
>> reusing cached zone  (/home/luca-linux/.cache/edera-kernel-bench/zone-6.18.bzImage)
>> reusing cached zone-tiny  (/home/luca-linux/.cache/edera-kernel-bench/zone-tiny-6.18.bzImage)

kernel->init  zone           ?s
kernel->init  zone-tiny      ?s

hyperfine command:
   hyperfine --shell=sh --warmup 2 --runs 100
    -n zone timeout 60 '/home/luca-linux/.cache/edera-kernel-bench/cloud-hypervisor' --kernel '/home/luca-linux/.cache/edera-kernel-bench/zone-6.18.bzImage' --initramfs '/tmp/tmp.VkGa9BrTBT/initrd.cpio.gz' --cmdline 'earlyprintk=ttyS0 console=ttyS0 quiet' --cpus boot=1 --memory size=512M --serial file='/dev/null' --console off >/dev/null 2>&1
    -n zone-tiny timeout 60 '/home/luca-linux/.cache/edera-kernel-bench/cloud-hypervisor' --kernel '/home/luca-linux/.cache/edera-kernel-bench/zone-tiny-6.18.bzImage' --initramfs '/tmp/tmp.VkGa9BrTBT/initrd.cpio.gz' --cmdline 'earlyprintk=ttyS0 console=ttyS0 quiet' --cpus boot=1 --memory size=512M --serial file='/dev/null' --console off >/dev/null 2>&1

Benchmark 1: zone
  Time (mean ± σ):     398.5 ms ±   7.3 ms    [User: 341.7 ms, System: 40.1 ms]
  Range (min … max):   386.3 ms … 426.6 ms    100 runs
 
Benchmark 2: zone-tiny
  Time (mean ± σ):     288.8 ms ±   5.3 ms    [User: 249.8 ms, System: 23.1 ms]
  Range (min … max):   278.7 ms … 308.8 ms    100 runs
 
Summary
  zone-tiny ran
    1.38 ± 0.04 times faster than zone

Closes RES-89

[tycho]

What's the tradeoff of keeping the ethernet NIC drivers in there? Those are the 2nd most likely case for PCI passthrough usage, and it feels a little awkward to keep those in a separate kernel config since they probably aren't that expensive to keep compared to all the other junk being removed.

[89luca89]

What's the tradeoff of keeping the ethernet NIC drivers in there? Those are the 2nd most likely case for PCI passthrough usage, and it feels a little awkward to keep those in a separate kernel config since they probably aren't that expensive to keep compared to all the other junk being removed.

mostly a shortcut because I don't really know what NIC we actually want to support (if we have a list?)
also I suppose most of the gains in boot time and size is skipping the module/squashfs thing

I was thinking, regular zone -> tiny-zone, regular zone + passthrough -> zone, gpu zone -> gpu flavour.

[tycho]

What's the tradeoff of keeping the ethernet NIC drivers in there? Those are the 2nd most likely case for PCI passthrough usage, and it feels a little awkward to keep those in a separate kernel config since they probably aren't that expensive to keep compared to all the other junk being removed.

mostly a shortcut because I don't really know what NIC we actually want to support (if we have a list?) also I suppose most of the gains in boot time and size is skipping the module/squashfs thing

We'd probably want to have these, excluding anything that is <1GbE:

• All Intel NICs (+ VF drivers)
• Broadcom: bnx2 (+ cnic), bnx2x, bnxt_en, bng_en
• All Mellanox NICs
• Amazon ENA adapter

In the future, we can enable any additional customer-requested NIC drivers too. But the above covers a wide variety of common deployments.

May also want to provide CONFIG_TUN, CONFIG_WIREGUARD, and CONFIG_VMXNET3 (VMware PV NIC).

I was thinking, regular zone -> tiny-zone, regular zone + passthrough -> zone, gpu zone -> gpu flavour.

I'd prefer that we keep the number of kernel variants we ship to a minimum. IMO the current split of zone + zone-gpu is heavily justified by the sheer size of the NVIDIA driver and its userspace dependencies. But the in-tree NIC drivers are tiny enough that we should just include them in all our zone kernels.

[89luca89]

We'd probably want to have these, excluding anything that is <1GbE:

• All Intel NICs (+ VF drivers)
• Broadcom: bnx2 (+ cnic), bnx2x, bnxt_en, bng_en
• All Mellanox NICs
• Amazon ENA adapter

In the future, we can enable any additional customer-requested NIC drivers too. But the above covers a wide variety of common deployments.

May also want to provide CONFIG_TUN, CONFIG_WIREGUARD, and CONFIG_VMXNET3 (VMware PV NIC).

Do we expect passthrough is nic only? At this point I suppose we need nvme stuff too?

[89luca89]

@tycho added back most stuff for nic and passthrough as of commit 1c718be

Still we get a measurable improvement in boot and size:

metric	zone	zone-tiny (PR)	zone-tiny (final)
bzImage (MB)	13.78	11.13	11.52
addons (MB)	6.80	0.05	5.59
total (MB)	20.60	11.20	17.14
smaller than zone	—	45%	16%
modules (=m)	55	0	~44
boot mean, 100 runs	396 ms	288.8 ms	296.9 ms
speedup vs zone	—	1.38×	1.33×

default_distrobox:linux-kernel-oci$ time ./hack/bench/compare-kernel-size.sh

FLAVOR           KERNEL     ADDONS    CONFIG      META   TOTAL(MB)
---------------------------------------------------------------------
zone             13.778      6.797     0.029     0.000      20.604
zone-tiny        11.521      5.590     0.024     0.000      17.135
delta zone -> zone-tiny:  3.470 MB smaller  (16%)

Benchmark 1: zone
Time (mean ± σ):     396.0 ms ±   7.2 ms    [User: 337.5 ms, System: 40.3 ms]
Range (min … max):   382.4 ms … 415.2 ms    100 runs
Benchmark 2: zone-tiny
Time (mean ± σ):     296.9 ms ±   6.9 ms    [User: 254.5 ms, System: 23.5 ms]
Range (min … max):   285.7 ms … 317.3 ms    100 runs
Summary
zone-tiny ran
    1.33 ± 0.04 times faster than zone

[bleggett]

Validated against protect's KVM and Xen e2e suites

That's all I'm looking for. If the tests don't need it, it doesn't matter. If the tests don't have it, and it does matter, then the tests need to be updated to validate it, and until then it doesn't matter. QED.