Upgrade to Josh r26.05.08 by christian-schilling · Pull Request #48 · rust-lang/josh-sync

christian-schilling · 2026-05-09T13:46:19Z

This is a first upgrade step that sets backward compatibility options. The adaption to new behaviour can be done in a later step.

Change: upgrade-josh

RalfJung · 2026-05-09T14:08:55Z

+                .into_owned()
+        })
+        .into_owned()
+}


This looks plausible but having a test that calls this function on a few representative inputs would probably still be a good idea.

RalfJung · 2026-05-09T14:14:58Z

I tried a "pull" sync on Miri with this and got an error:

Updating/installing josh-proxy binary...
+ cd "/home/r/src/rust/miri" && "cargo" "+stable" "install" "--locked" "--git" "https://github.com/josh-project/josh" "--tag" "r26.05.08" "josh-proxy"
+ cd "/home/r/src/rust/miri" && "git" "ls-remote" "https://github.com/rust-lang/rust" "HEAD"
+ cd "/home/r/src/rust/miri" && "git" "status" "--untracked-files=no" "--porcelain"
josh up and running
+ cd "/home/r/src/rust/miri" && "git" "rev-parse" "HEAD"
previous upstream base: 63b1dfc0e00fd6f8ad7cd8817fc712e7d9b7be59
new upstream base: 0490dd938541ad996c5ad1ec6e274012afe3e1d4
original local HEAD: dd8994ddd4f5dc8564353854f5ea8513aa81807f
+ cd "/home/r/src/rust/miri" && "git" "add" "rust-version"
+ cd "/home/r/src/rust/miri" && "git" "commit" "rust-version" "--no-verify" "-m" "Prepare for merging from rust-lang/rust\n\nThis updates the rust-version file to 0490dd938541ad996c5ad1ec6e274012afe3e1d4."
+ cd "/home/r/src/rust/miri" && "git" "fetch" "http://localhost:42042/rust-lang/rust.git@0490dd938541ad996c5ad1ec6e274012afe3e1d4:~(history=\"keep-trivial-merges\",gpgsig=\"norm-lf\")[:rev(<=75dd959a3a40eb5b4574f8d2e23aa6efbeb33573:prefix=src/tools/miri):/src/tools/miri].git"
Reverting HEAD to dd8994ddd4f5dc8564353854f5ea8513aa81807f
+ cd "/home/r/src/rust/miri" && "git" "reset" "--hard" "dd8994ddd4f5dc8564353854f5ea8513aa81807f"
I have to kill josh-proxy the hard way, let's hope this does not break anything.
Pull failure: cannot fetch git state through Josh

Caused by:
    Command `cd "/home/r/src/rust/miri" && "git" "fetch" "http://localhost:42042/rust-lang/rust.git@0490dd938541ad996c5ad1ec6e274012afe3e1d4:~(history=\"keep-trivial-merges\",gpgsig=\"norm-lf\")[:rev(<=75dd959a3a40eb5b4574f8d2e23aa6efbeb33573:prefix=src/tools/miri):/src/tools/miri].git"` failed with exit code Some(128). STDOUT:
    
    STDERR:
    fatal: unable to access 'http://localhost:42042/rust-lang/rust.git@0490dd938541ad996c5ad1ec6e274012afe3e1d4:~(history="keep-trivial-merges",gpgsig="norm-lf")[:rev(<=75dd959a3a40eb5b4574f8d2e23aa6efbeb33573:prefix=src/tools/miri):/src/tools/miri].git/': The requested URL returned error: 400

This is a first upgrade step that sets backward compatibility options. The adaption to new behaviour can be done in a later step. Since setting the options for the filter requires quotes in the filter, this also changes URL construction to always encode special characters using urlencode. This is safer anyway. Change: upgrade-josh

christian-schilling · 2026-05-09T17:12:11Z

The quoted strings in the url where the problem. Fixed it by always encoding the filter using urlencode.

RalfJung · 2026-05-09T19:30:58Z

Never mind I compiled the wrong version of this PR.^^

RalfJung · 2026-05-09T19:54:16Z

I'm still getting "I have to kill josh-proxy the hard way, let's hope this does not break anything", which I don't think I got before. It seems like stopping josh-proxy does not work any more the way it used to work? What we do is we send SIGINT, wait 100ms, and then see if the process is gone. If it is not, we send SIGKILL and emit this message.
(#49 does not help so it seems like SIGINT just doesn't do anything any more.)

Other than that, I did a "pull" which worked (but there were no changes so this is not a full test). The resulting PR looks reasonable.
I also did a "push". That took very long (it was after the pull so it should have already had a reasonably up-to-date cache -- or maybe due to josh not being stopped properly the cache was invalid). But other than that it worked fine as well, the resulting PR looks reasonable.

RalfJung · 2026-05-09T20:03:47Z

I can reproduce the shutdown problems outside of the sync tool: when I run

/home/r/.cargo/bin/josh-proxy "--local" "/home/r/.cache/rustc-josh" "--remote=https://github.com" "--port=42042" "--no-background"

and then wait a bit and do Ctrl-C, all I get is

^Cshutdown_signal
shutdown requested

but it keeps running. I can hit Ctrl-C as often as I want, nothing happens.

RalfJung · 2026-05-10T08:32:05Z

Sending SIGTERM instead seems to work. But I don't know whether that leads to a clean shutdown -- there's no shutdown requested from josh when I do that.

christian-schilling · 2026-05-10T12:29:08Z

I think SIGTERM is also fine. One good thing about how the git odb works is that it does not get corrupted by something like that.
Also I would not spend to much effort on handling the proxy child process lifecycle beyond getting it to work somehow. My plan is to next work on a PR that replaces the usage of josh-proxy with the new josh cli tool. That will avoid those kind of problems entirely.

RalfJung · 2026-05-10T12:34:10Z

It still seems like a bug that josh-proxy apparently receives the Ctrl-C and tries to shut down... but then doesn't actually quit.

RalfJung · 2026-05-10T12:46:31Z

But, okay, we can just merge #49 then to deal with the shutdown issue.

What about the observation that "push" has gotten a lot slower? I did the exact same push with old and new josh (just after a pull, to ensure a hot cache)

old josh (7e74618): 20s
new josh (50f9e96): 83s

christian-schilling · 2026-05-10T13:26:33Z

It still seems like a bug that josh-proxy apparently receives the Ctrl-C and tries to shut down... but then doesn't actually quit.

True. josh-project/josh#2004

christian-schilling · 2026-05-10T13:27:40Z

But, okay, we can just merge #49 then to deal with the shutdown issue.

What about the observation that "push" has gotten a lot slower? I did the exact same push with old and new josh (just after a pull, to ensure a hot cache)
* old josh ([7e74618](https://github.com/rust-lang/josh-sync/commit/7e74618efc1838da65c111afd3053c0bead19744)): 20s

* new josh ([50f9e96](https://github.com/rust-lang/josh-sync/commit/50f9e96740dfffdc97609041b006a88690b31f8f)): 83s

Hm, strange. It should be faster, not slower. Will investigate what's going on.

christian-schilling · 2026-05-13T05:53:35Z

According to our tests, miri push perf should be vastly better with
josh-project/josh#2005

christian-schilling force-pushed the @heads/main/christian.schilling.de@gmail.com branch from a8a3926 to 28f8cdb Compare May 9, 2026 13:50

christian-schilling mentioned this pull request May 9, 2026

Upgrading Josh #42

Open

RalfJung reviewed May 9, 2026

View reviewed changes

christian-schilling force-pushed the @heads/main/christian.schilling.de@gmail.com branch from 28f8cdb to 50f9e96 Compare May 9, 2026 17:10

This comment was marked as outdated.

Sign in to view

Conversation

christian-schilling commented May 9, 2026

Uh oh!

RalfJung May 9, 2026

Choose a reason for hiding this comment

Uh oh!

christian-schilling May 9, 2026

Choose a reason for hiding this comment

Uh oh!

RalfJung commented May 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

christian-schilling commented May 9, 2026

Uh oh!

This comment was marked as outdated.

RalfJung commented May 9, 2026

Uh oh!

RalfJung commented May 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RalfJung commented May 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RalfJung commented May 10, 2026

Uh oh!

christian-schilling commented May 10, 2026

Uh oh!

RalfJung commented May 10, 2026

Uh oh!

RalfJung commented May 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

christian-schilling commented May 10, 2026

Uh oh!

christian-schilling commented May 10, 2026

Uh oh!

christian-schilling commented May 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

RalfJung commented May 9, 2026 •

edited

Loading

RalfJung commented May 9, 2026 •

edited

Loading

RalfJung commented May 9, 2026 •

edited

Loading

RalfJung commented May 10, 2026 •

edited

Loading