Skip to content

Fix curl with c-ares failing to resolve DNS inside sandbox on macOS #14792

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
Enzime wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Fix curl with c-ares failing to resolve DNS inside sandbox on macOS #14792

Enzime wants to merge 1 commit into from
+3 −1

Conversation

@Enzime
Copy link
Member

@Enzime Enzime commented Dec 15, 2025

Motivation

Currently on Nixpkgs master, curl is built with c-ares which causes DNS resolution to fail when FODs are built inside the Nix build sandbox on macOS.

The change (NixOS/nixpkgs#451579) has already been reverted in staging (NixOS/nixpkgs#462692) but I still think this PR is useful as we still want FODs that use c-ares to build on macOS.

Context

When I try to build the following FOD without this PR, I get:

$ cat flake.nix
{
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/23735a82a828372c4ef92c660864e82fbe2f5fbe";

  outputs = { self, nixpkgs }:
    let
      system = "aarch64-darwin";
      pkgs = nixpkgs.legacyPackages.${system};
    in
    {
      packages.${system}.default = pkgs.stdenvNoCC.mkDerivation {
        name = "example-fod";

        outputHash = "";
        outputHashAlgo = "sha256";
        outputHashMode = "flat";

        nativeBuildInputs = [ pkgs.curl pkgs.cacert ];

        dontUnpack = true;

        buildPhase = ''
          curl -v --max-time 10 https://cache.nixos.org/nix-cache-info -o $out 2>&1
        '';

        dontInstall = true;
      };
    };
}
$ nix build --print-build-logs
warning: found empty hash, assuming 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
example-fod> Running phase: patchPhase
example-fod> Running phase: updateAutotoolsGnuConfigScriptsPhase
example-fod> Running phase: configurePhase
example-fod> no configure script, doing nothing
example-fod> Running phase: buildPhase
example-fod>   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
example-fod>                                  Dload  Upload   Total   Spent    Left  Speed
example-fod>   0     0   0     0   0     0     0     0  --:--:-- --:--:-- --:--:--     0* Could not resolve host: cache.nixos.org (Could not contact DNS servers)
example-fod> * Store negative name resolve for cache.nixos.org:443
example-fod> * shutting down connection #0
example-fod> curl: (6) Could not resolve host: cache.nixos.org (Could not contact DNS servers)

With this PR:

$ nix build --print-build-logs
warning: found empty hash, assuming 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
example-fod> Running phase: patchPhase
example-fod> Running phase: updateAutotoolsGnuConfigScriptsPhase
example-fod> Running phase: configurePhase
example-fod> no configure script, doing nothing
example-fod> Running phase: buildPhase
example-fod>   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
example-fod>                                  Dload  Upload   Total   Spent    Left  Speed
example-fod>   0     0   0     0   0     0     0     0  --:--:-- --:--:-- --:--:--     0* Host cache.nixos.org:443 was resolved.
example-fod> * IPv6: 2a04:4e42::347, 2a04:4e42:200::347, 2a04:4e42:400::347, 2a04:4e42:600::347
example-fod> * IPv4: 151.101.1.91, 151.101.65.91, 151.101.129.91, 151.101.193.91
example-fod> *   Trying [2a04:4e42::347]:443...
example-fod> * Immediate connect fail for 2a04:4e42::347: No route to host
example-fod> *   Trying 151.101.1.91:443...
example-fod> * ALPN: curl offers h2,http/1.1
example-fod> } [5 bytes data]
example-fod> * TLSv1.3 (OUT), TLS handshake, Client hello (1):
example-fod> } [1559 bytes data]
example-fod> * SSL Trust Anchors:
example-fod> *   CAfile: /nix/store/z6hzprzrkqj48m9kk4xkzfs6v9hmrylj-nss-cacert-3.115/etc/ssl/certs/ca-bundle.crt
example-fod> { [5 bytes data]
example-fod> * TLSv1.3 (IN), TLS handshake, Server hello (2):
example-fod> { [1210 bytes data]
example-fod> * TLSv1.3 (IN), TLS change cipher, Change cipher spec (1):
example-fod> { [1 bytes data]
example-fod> * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
example-fod> { [19 bytes data]
example-fod> * TLSv1.3 (IN), TLS handshake, Certificate (11):
example-fod> { [2584 bytes data]
example-fod> * TLSv1.3 (IN), TLS handshake, CERT verify (15):
example-fod> { [264 bytes data]
example-fod> * TLSv1.3 (IN), TLS handshake, Finished (20):
example-fod> { [36 bytes data]
example-fod> * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
example-fod> } [1 bytes data]
example-fod> * TLSv1.3 (OUT), TLS handshake, Finished (20):
example-fod> } [36 bytes data]
example-fod> * SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / X25519MLKEM768 / RSASSA-PSS
example-fod> * ALPN: server accepted h2
example-fod> * Server certificate:
example-fod> *   subject: CN=cache.nixos.org
example-fod> *   start date: Nov 14 02:53:59 2025 GMT
example-fod> *   expire date: Feb 12 02:53:58 2026 GMT
example-fod> *   issuer: C=US; O=Let's Encrypt; CN=R13
example-fod> *   Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
example-fod> *   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
example-fod> *   Certificate level 2: Public key type RSA (4096/152 Bits/secBits), signed using sha256WithRSAEncryption
example-fod> *   subjectAltName: "cache.nixos.org" matches cert's "cache.nixos.org"
example-fod> * SSL certificate verified via OpenSSL.
example-fod> * Established connection to cache.nixos.org (151.101.1.91 port 443) from 192.168.1.157 port 49660
example-fod> * using HTTP/2
example-fod> * [HTTP/2] [1] OPENED stream for https://cache.nixos.org/nix-cache-info
example-fod> * [HTTP/2] [1] [:method: GET]
example-fod> * [HTTP/2] [1] [:scheme: https]
example-fod> * [HTTP/2] [1] [:authority: cache.nixos.org]
example-fod> * [HTTP/2] [1] [:path: /nix-cache-info]
example-fod> * [HTTP/2] [1] [user-agent: curl/8.17.0]
example-fod> * [HTTP/2] [1] [accept: */*]
example-fod> } [5 bytes data]
example-fod> > GET /nix-cache-info HTTP/2
example-fod> > Host: cache.nixos.org
example-fod> > User-Agent: curl/8.17.0
example-fod> > Accept: */*
example-fod> >
example-fod> { [5 bytes data]
example-fod> * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
example-fod> { [201 bytes data]
example-fod> * Request completely sent off
example-fod> { [5 bytes data]
example-fod> < HTTP/2 200
example-fod> < last-modified: Fri, 03 Sep 2021 14:43:34 GMT
example-fod> < etag: "69d68de32bd6e6f0751e4cd5ae13e053"
example-fod> < content-type: text/x-nix-cache-info
example-fod> < server: AmazonS3
example-fod> < via: 1.1 varnish, 1.1 varnish
example-fod> < x-timer: S1696362712.176032,VS0,VS0,VE0
example-fod> < accept-ranges: bytes
example-fod> < date: Mon, 15 Dec 2025 03:13:46 GMT
example-fod> < age: 136580
example-fod> < x-served-by: cache-iad-kjyo7100111-IAD, cache-fra-etou8220181-FRA
example-fod> < x-cache: HIT, HIT
example-fod> < x-cache-hits: 31681, 11
example-fod> < access-control-allow-origin: *
example-fod> < content-length: 51
example-fod> <
example-fod> { [51 bytes data]
example-fod> 100    51 100    51   0     0   892     0  --:--:-- --:--:-- --:--:--   910
example-fod> * Connection #0 to host cache.nixos.org:443 left intact
example-fod> Running phase: fixupPhase
example-fod> checking for references to /nix/var/nix/builds/nix-57284-3318981716/ in /nix/store/27rjmapda174yqs67vzwppdb6gkr98wj-example-fod...
example-fod> patching script interpreter paths in /nix/store/27rjmapda174yqs67vzwppdb6gkr98wj-example-fod
error: hash mismatch in fixed-output derivation '/nix/store/9pjc3i2v2zci1shrizpf4h1sw8bzdb9w-example-fod.drv':
         specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
            got:    sha256-LJ3jc651pScWN2NQNERaXNOmrjWsbDBtQMDgZ2R4WJc=

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

@Enzime Enzime requested a review from Ericson2314 as a code owner December 15, 2025 03:21
@Enzime Enzime mentioned this pull request Dec 15, 2025
Open
3 tasks
Enzime
Enzime commented Dec 16, 2025
View reviewed changes
src/libstore/unix/build/sandbox-network.sb

; Allow DNS lookups.
(allow network-outbound (remote unix-socket (path-literal "/private/var/run/mDNSResponder")))
(allow mach-lookup (global-name "com.apple.SystemConfiguration.DNSConfiguration"))
Copy link
Member Author

@Enzime Enzime Dec 16, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Mic92 it seems like you commented on the commits so they don't show up at all in PR view as far as I can tell

@Mic92:

Can you explain what line does
Is it allowing all micro kernel syscalls that are sent to the dns service?

I'm not 100% sure if they're syscalls, my understanding is that c-ares uses the SystemConfiguration.framework and that's why it needs that sandbox permission:

https://github.com/c-ares/c-ares/blob/16c873c2e58325a1872ff85881284357c15c9084/src/lib/ares_sysconfig_mac.c#L91-L94

The way I figured out what sandbox permissions were missing from this profile by running:

$ nix build --print-build-logs --option darwin-log-sandbox-violations true
...


$ /usr/bin/log show --predicate 'eventMessage CONTAINS "deny" AND eventMessage CONTAINS "curl"' --last 2m
Filtering the log data using "composedMessage CONTAINS "deny" AND composedMessage CONTAINS "curl""
Skipping info and debug messages, pass --info and/or --debug to include.
Timestamp                       Thread     Type        Activity             PID    TTL
2025-12-16 15:47:29.803630+0100 0xa711a    Error       0x0                  0      0    kernel: (Sandbox) Sandbox: curl(44985) deny(1) file-read-metadata /System/Cryptexes/OS
2025-12-16 15:47:29.809728+0100 0xa711a    Error       0x0                  0      0    kernel: (Sandbox) Sandbox: curl(44985) deny(1) file-read-data /private/etc/passwd
2025-12-16 15:47:29.809737+0100 0xa711a    Error       0x0                  0      0    kernel: (Sandbox) Sandbox: curl(44985) deny(1) file-read-metadata /private/etc/passwd
2025-12-16 15:47:29.809768+0100 0xa683b    Error       0x0                  0      0    kernel: (Sandbox) Sandbox: curl(44985) deny(1) mach-lookup com.apple.system.notification_center
2025-12-16 15:47:29.809866+0100 0xa683b    Error       0x0                  0      0    kernel: (Sandbox) Sandbox: curl(44985) deny(1) mach-lookup com.apple.logd
2025-12-16 15:47:29.810242+0100 0xa711a    Error       0x0                  0      0    kernel: (Sandbox) Sandbox: curl(44985) deny(1) file-read-data /dev/autofs_nowait
2025-12-16 15:47:29.810267+0100 0xa711a    Error       0x0                  0      0    kernel: (Sandbox) Sandbox: curl(44985) deny(1) file-read-data /private/etc/passwd
2025-12-16 15:47:29.810272+0100 0xa711a    Error       0x0                  0      0    kernel: (Sandbox) Sandbox: curl(44985) deny(1) file-read-metadata /private/etc/passwd
2025-12-16 15:47:29.810367+0100 0xa711a    Error       0x0                  0      0    kernel: (Sandbox) Sandbox: curl(44985) deny(1) file-read-data /dev/autofs_nowait
2025-12-16 15:47:29.812502+0100 0xa711a    Error       0x0                  0      0    kernel: (Sandbox) Sandbox: curl(44985) deny(1) file-read-metadata /System/Cryptexes/App
2025-12-16 15:47:29.812506+0100 0xa711a    Error       0x0                  0      0    kernel: (Sandbox) Sandbox: curl(44985) deny(1) file-read-metadata /System/Cryptexes/OS
2025-12-16 15:47:29.812519+0100 0xa711a    Error       0x0                  0      0    kernel: (Sandbox) Sandbox: curl(44985) deny(1) file-read-data /System/Library/Preferences/Logging/Subsystems/com.apple.xpc.plist
2025-12-16 15:47:29.812538+0100 0xa683b    Error       0x0                  0      0    kernel: (Sandbox) Sandbox: curl(44985) deny(1) mach-lookup com.apple.SystemConfiguration.DNSConfiguration
2025-12-16 15:47:29.812612+0100 0xa711a    Error       0x0                  0      0    kernel: (Sandbox) Sandbox: curl(44985) deny(1) file-read-metadata /private/etc/hosts
2025-12-16 15:47:29.812627+0100 0xa711a    Error       0x0                  0      0    kernel: (Sandbox) Sandbox: curl(44985) deny(1) file-read-data /private/etc/hosts

And then I tried different sets of sandbox rules to get it working

We also have some (allow mach-lookup ...) lines at the bottom of this file for trustd

Copy link
Member

@Mic92 Mic92 Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't loading the library itself is issue. It looks like com.apple.SystemConfiguration.DNSConfiguration is a so called mach service.

Copy link
Member

@Mic92 Mic92 Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I just learned there are Mach Ports, which is an kernel-managed message queue and processes can register well-known names for those. When a process does a mach-lookup it receives the right to communicate to this port. In this case the dns configuration service.

Copy link
Member Author

@Enzime Enzime Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c-ares uses the dns_configuration_copy function which requires the com.apple.SystemConfiguration.DNSConfiguration service I believe

https://github.com/c-ares/c-ares/blob/16c873c2e58325a1872ff85881284357c15c9084/src/lib/ares_sysconfig_mac.c#L118-L119

https://github.com/c-ares/c-ares/blob/16c873c2e58325a1872ff85881284357c15c9084/src/lib/ares_sysconfig_mac.c#L337

The function is defined here:

https://github.com/apple-open-source/macos/blob/f60c49b581c54dfc3367b3ebc5b7904fd7d5db7b/configd/dnsinfo/dnsinfo_copy.c#L83

Mic92
Mic92 reviewed Dec 16, 2025
View reviewed changes
src/libstore/unix/build/sandbox-network.sb
(allow file-read*
(literal "/private/var/run/resolv.conf"))
(literal "/private/var/run/resolv.conf")
(literal "/private/etc/hosts"))
Copy link
Member

@Mic92 Mic92 Dec 16, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this needed. This seems unrelated to DNS?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Mic92 Mic92 Mic92 left review comments

@Ericson2314 Ericson2314 Awaiting requested review from Ericson2314 Ericson2314 is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Enzime @Mic92