Usage in NixOS

[lovesegfault]

Hi!

I'm working on a new approach to kernel builds for the NixOS project and wanted to try using autokernel, but have been struggling to make it work.

Firstly, I cannot get the integration tests to pass:

❯ cargo test --test integration_tests -- --nocapture integration_setup
    Finished test [unoptimized + debuginfo] target(s) in 0.07s
     Running tests/integration_tests.rs (target/debug/deps/integration_tests-718890ec49e56c32)

running 1 test
creating /tmp/autokernel-test directory
kernel tar already in cache
extracting kernel linux-5.19.1 ...
make: /tmp/autokernel-test/linux-5.19.1/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
make: /tmp/autokernel-test/linux-5.19.1/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
make: /tmp/autokernel-test/linux-5.19.1/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
make: /tmp/autokernel-test/linux-5.19.1/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
make: /tmp/autokernel-test/linux-5.19.1/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
make: /tmp/autokernel-test/linux-5.19.1/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
make: /tmp/autokernel-test/linux-5.19.1/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
make: /tmp/autokernel-test/linux-5.19.1/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
make: /tmp/autokernel-test/linux-5.19.1/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
make: /tmp/autokernel-test/linux-5.19.1/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
make: /tmp/autokernel-test/linux-5.19.1/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
make: /tmp/autokernel-test/linux-5.19.1/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
make: /tmp/autokernel-test/linux-5.19.1/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
Makefile:622: arch//Makefile: No such file or directory
make: *** No rule to make target 'arch//Makefile'.  Stop.
thread 'integration_setup' panicked at 'called `Result::unwrap()` on an `Err` value: Could not prepare bridge in /tmp/autokernel-test/linux-5.19.1

Caused by:
    Condition failed: `builder_output.status.success()`', tests/setup_teardown.rs:70:23
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
test integration_setup ... FAILED

failures:

failures:
    integration_setup

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 3 filtered out; finished in 5.47s

error: test failed, to rerun pass `--test integration_tests`

This same problem is reflected when I try to use autokernel for our kernel builds, which fail:

linux-config> make: /build/linux-4.14.306/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
linux-config> make: /build/linux-4.14.306/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
linux-config> make: /build/linux-4.14.306/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
linux-config> make: /build/linux-4.14.306/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
linux-config> make: /build/linux-4.14.306/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
linux-config> make: /build/linux-4.14.306/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
linux-config> make: /build/linux-4.14.306/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
linux-config> make: /build/linux-4.14.306/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
linux-config> make: /build/linux-4.14.306/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
linux-config> make: /build/linux-4.14.306/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
linux-config> make: /build/linux-4.14.306/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
linux-config> make: /build/linux-4.14.306/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
linux-config> make: /build/linux-4.14.306/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
linux-config> make: /build/linux-4.14.306/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
linux-config> make: /build/linux-4.14.306/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
linux-config> make: /build/linux-4.14.306/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
linux-config> make: /build/linux-4.14.306/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
linux-config> make: /build/linux-4.14.306/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
linux-config> make: /build/linux-4.14.306/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
linux-config> make: /build/linux-4.14.306/scripts/kconfig/autokernel_interceptor.sh: No such file or directory
linux-config> Makefile:516: arch//Makefile: No such file or directory
linux-config> make: *** No rule to make target 'arch//Makefile'.  Stop.
linux-config> error: Could not prepare bridge in .
linux-config> because: Condition failed: `builder_output.status.success()`

[oddlama]

Hi! I'm very humbled that you'd attempt to use autokernel for the NixOS build process. Sorry for the inconveniences, the integration tests are currently a little rough around the edges. The tests are currently assuming the existence of some system utilities, so there are some useless error messages produced if something mandatory isn't found. I created this tool just before I switched to NixOS myself, so there are some dependencies left that are not properly documented right now. _{(I didn't notice them on gentoo previously for obvious reasons)}

About your issue: You are encountering the No such file or directory error not because the file doesn't exist, but because the #!/bin/bash shebang inside of this interceptor (and therefore all other kernel scripts) cannot be resolved by your system environment. I'm not sure what the correct way is to get this working on NixOS, but the hacky way is to just link /bin/bash to the correct bash binary in the nix store.

Generally I tried to derive all required tools for building the kernel from the upstream definition here and arrived at the following commands:

> nix shell nixpkgs#{gcc,bash,gnumake,perl,gmp,libmpc,mpfr,bison,flex,pahole,python3,wget,tar}
> ln -s /nix/store/...bash-5.2-p15/bin/bash /bin/bash # Very hacky, sorry :(
> cargo test --test integration_tests -- --nocapture integration_setup
    Finished test [unoptimized + debuginfo] target(s) in 0.10s
     Running tests/integration_tests.rs (target/debug/deps/integration_tests-84e17e38762193f8)

running 1 test
creating /tmp/autokernel-test directory
kernel tar already in cache
extracting kernel linux-5.19.1 ...
       Built bridge for /tmp/autokernel-test/linux-5.19.1 in 6.34s
 Initialized bridge [kernel 5.19.1, 16885 symbols] in 2.30s
test integration_setup ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 3 filtered out; finished in 16.46s

_{(wget and tar are required for the integration test, but it seems like you have those already. Also /tmp must allow executable files at the moment, not sure whether that might become a problem in NixOS build sandbox)}

I guess I should probably switch to a flake based build for this project to avoid the issues, but I'm not yet entirely sure what the best way would be to do that. I hope this solves your issue, if you need additional information for your integration attempt I'm of course glad to help!

[lovesegfault]

Ah, you want that shebang to be #!/usr/bin/env bash, which guarantees it works everywhere.

Also, if you are interested, I'm working on the new kernel build infrastructure here: https://github.com/lovesegfault/nixpkgs-kernel-redux

The idea is to translate a Nix module into an autokernel config, and then use that to generate the final config file. You can see that being done here: https://github.com/lovesegfault/nixpkgs-kernel-redux/blob/main/kconfig/default.nix

One thing that autokernel could do to help us is allow passing a defconfig name instead of a path. That is, I can tell it to use whatever_defconfig as my base, since unfortunately, it's the way the information is recorded in nixpkgs: https://github.com/NixOS/nixpkgs/blob/master/lib/systems/platforms.nix#L29

[lovesegfault]

(I'll submit a PR packaging this with a flake, I already did most of the work here: https://github.com/lovesegfault/nixpkgs-kernel-redux/blob/main/kconfig/autokernel.nix)

[oddlama]

Ah, you want that shebang to be #!/usr/bin/env bash, which guarantees it works everywhere.

True, that's probably a better way to do it. Do you know what nixos is doing to make the kernel scripts work? AFAIK those all start with #!/bin/bash too.

Also, if you are interested, I'm working on the new kernel build infrastructure here: https://github.com/lovesegfault/nixpkgs-kernel-redux

The idea is to translate a Nix module into an autokernel config, and then use that to generate the final config file. You can see that being done here: https://github.com/lovesegfault/nixpkgs-kernel-redux/blob/main/kconfig/default.nix

Awesome, thanks for the links!

One thing that autokernel could do to help us is allow passing a defconfig name instead of a path. That is, I can tell it to use whatever_defconfig as my base, since unfortunately, it's the way the information is recorded in nixpkgs: https://github.com/NixOS/nixpkgs/blob/master/lib/systems/platforms.nix#L29

Yep, currently autokernel just accepts a path without any logic behind it like:

load_kconfig_unchecked(kernel_dir .. "/arch/x86/configs/x86_64_defconfig")

I didn't want to put special logic into autokernel itself to figure out the correct path, since it's not directly obvious what it would be and if it would change in the future. Just have a quick look at the following examples:

> fd -g defconfig arch/
arch/alpha/configs/defconfig
arch/arm64/configs/defconfig
arch/csky/configs/defconfig
arch/riscv/configs/defconfig
arch/s390/configs/defconfig

> fd x86_64_defconfig
arch/um/configs/x86_64_defconfig
arch/x86/configs/x86_64_defconfig

Since autokernel has access to the build-time environment variables, we can derive the location automatically: In the Makefile it is defined as $(SRCTREE)/arch/$(SRCARCH)/configs/$(KBUILD_DEFCONFIG). I've just pushed a small addition to the lua interface that allows accessing the kernel environment variables so you can now do the following:

load_kconfig_unchecked(kernel_dir .. "/arch/" .. ak.kernel_env("SRCARCH") .. "/configs/" .. ak.kernel_env("KBUILD_DEFCONFIG"))

Does that solve your problem?

---

On a related note: How do you feel about using lua to do the config? For other distributions I felt this is a good compromise between being easy to read and configure while still having access to more powerful functionality, but for nix I'm not entirely sure whether lua is the best language for the job. The whole scripting system is already entirely decoupled from autokernel's main functionality, so if you have an idea for a better config format / language it wouldn't be too much work to get that in.

[Janik-Haag]

We could use https://github.com/nix-community/patsh for patching the shell stuff, it's already used in a few other nipxkgs.