Skip to content

Start Docker early and retry or fail-fast instead of after building wheels - #10283

Merged
hugovk merged 1 commit into
python-pillow:mainfrom
hugovk:wheels-docker-daemon
Oct 23, 2026
Merged

hugovk merged 1 commit into
python-pillow:mainfrom
hugovk:wheels-docker-daemon

Conversation

@hugovk

@hugovk hugovk commented Oct 22, 2026

Copy link
Copy Markdown
Member

Windows AMD64 and x86 often fail to connect to the Docker daemon on the wheels build:

  + docker run --rm -v D:\a\Pillow\Pillow:C:\pillow -v C:\cibw:C:\cibw -v %CD%\..\venv-test:%CD%\..\venv-test -e CI -e GITHUB_ACTIONS -e FORCE_COLOR mcr.microsoft.com/windows/servercore:ltsc2022 powershell C:\pillow\.github\workflows\wheels-test.ps1 %CD%\..\venv-test
  failed to connect to the docker API at npipe:////./pipe/docker_engine; check if the path is correct and if the daemon is running: open //./pipe/docker_engine: The system cannot find the file specified.
Error: cibuildwheel: Command docker run --rm -v D:\a\Pillow\Pillow:C:\pillow -v C:\cibw:C:\cibw -v %CD%\..\venv-test:%CD%\..\venv-test -e CI -e GITHUB_ACTIONS -e FORCE_COLOR mcr.microsoft.com/windows/servercore:ltsc2022 powershell C:\pillow\.github\workflows\wheels-test.ps1 %CD%\..\venv-test failed with code 1. 

https://github.com/python-pillow/Pillow/actions/runs/38960170632/job/117173653596

And this happens when cibuildwheel tests the first wheel it builds, so the job dies about 10 minutes in with 1 of 9 wheels done.

In the last 20 wheel runs [at the time of writing, last week], the Windows jobs failed 14 times, 9 on AMD64 and 5 on x86. Eleven of those were this failure to connect, including every one of the x86 failures.

And it just happened on https://github.com/python-pillow/Pillow/actions/runs/39778421171/job/119825457340?pr=10282 [today].

I don't know why it happens, but the idea of this PR is to attempt to connect to Docker sooner, and retry on failure. If it's still going to fail, it will fail sooner. We'd still need to wait until other jobs finish to click restart, but this will at least free up a runner for other jobs.

@radarhere radarhere left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've run this "Start Docker" step two dozen times in my fork, and it works on the first attempt every time. So I suspect that Start-Service docker is really all that is needed here - but if you want to play it safe, then go ahead.

@hugovk

hugovk commented Oct 23, 2026

Copy link
Copy Markdown
Member Author

Yeah, I started this PR after it failed during the release build last week, then ran it 10 times on my fork over the past week, and all were fine first time.

But then I saw it fail in the linked run yesterday.

I think pass first time means the daemon was already up and running, so nothing to actually fix in those cases. Let's keep the loop to hopefully help when it does fail.

@hugovk
hugovk merged commit b8dba50 into python-pillow:main Oct 23, 2026
48 of 49 checks passed
@hugovk
hugovk deleted the wheels-docker-daemon branch October 23, 2026 08:07
@radarhere

Copy link
Copy Markdown
Member

To clarify a little - I meant that I think

- name: Start Docker
  if: matrix.cibw_arch != 'ARM64'
  run: |
    Start-Service docker
  shell: pwsh

would be sufficient.

But whatever.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants