Drastically restrict the grammar of tuple indices

[fmease]

Syntactically speaking, on stable & main,

1. tuple indices in struct exprs & pats are simply unsuffixed integer literals
   • meaning they adhere to [0-9][0-9_]*|0b[01][01_]*|0o[0-7][0-7_]*|0x[0-9a-fA-F][0-9a-fA-F_]*
   • we accept for example X { 00: x, 1_0: y, 0xF: z }
   • we (thankfully) reject e.g., X { 1e2: () }, X { 1e+2: () } and X { 1suffix: () }
2. tuple indices in field & offset_of exprs and field_of types are either unsuffixed integer literals or a subset of unsuffixed float literals
   • they adhere to [0-9][0-9_]*([eE]_*[0-9][0-9_]*)|0b[01][01_]*|0o[0-7][0-7_]*|0x[0-9a-fA-F][0-9a-fA-F_]*
   • we accept for example x.00, x.1_0, x.0xF and x.1e2
   • we (thankfully) reject e.g., x.1e+2 and x.0suffix
3. tuple index shorthands like S { 0 } are legal in patterns but not in expressions
   • they were made grammatical in PR RUST-81235 (2021) w/o lang FCP in order to improve diagnostics
   • clearly, they should not be syntactically legal
   • fixed in separate PR: Syntactically reject tuple index shorthands in struct patterns to fix a correctness regression #155698

I find this part of the grammar a bit unfortunate and I'd like to drastically restrict it to just 0|[1-9][0-9]* (i.e., plain non-negative decimal numbers without leading zeroes) with this PR. I hope it's obvious why that's desirable.

Almost decade ago (still way after 1.0) we actually used to reject numeric bases, underscores and leading zeroes in tuple indices (leading zeroes specifically since PR RUST-47084 (2017)) but then that was undone by PR RUST-49718 (2018) w/o lang FCP in order to simplify the compiler. Moreover, we've been rejecting suffixes on them since at least 1.0¹.

Of course in Rust tuple indices aren't first class citizens and are closer to alphanumeric identifiers than proper natural numbers. Still, they're clearly meant to mimic natural numbers (or at least they used to). That's why I find it odd that 00 is legal and unequal to 0, similarly with 1_000 and 1000. By rejecting leading zeroes and underscores we can "keep up the appearance that these indices are natural numbers instead of identifiers" which I quite like.

Footnotes

1. With some hiccups, see RUST-59418, RUST-60186 and RUST-145463. ↩

[fmease]

@bors try @rust-timer queue

[rust-bors]

☀️ Try build successful (CI)
Build commit: 53f0b05 (53f0b05ecbcf011a1f77f1c627304e87ebd05801, parent: e1613686e0efc80a1a18b1263625450a8de3fb04)

[rust-timer]

Finished benchmarking commit (53f0b05): comparison URL.

Overall result: no relevant changes - no action needed

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results (primary 11.4%, secondary -6.2%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	11.4%	[11.4%, 11.4%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-6.2%	[-6.2%, -6.2%]	1
All ❌✅ (primary)	11.4%	[11.4%, 11.4%]	1

Cycles

Results (primary -2.3%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-2.3%	[-2.3%, -2.3%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-2.3%	[-2.3%, -2.3%]	1

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 484.178s -> 486.044s (0.39%)
Artifact size: 395.05 MiB -> 395.09 MiB (0.01%)

[fmease]

@craterbot check

[craterbot]

👌 Experiment pr-154492 created and queued.
🤖 Automatically detected try build 53f0b05
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[petrochenkov]

I hope it's obvious why that's desirable.

Not really, I still think 49718 was generally right, instead of introducing a separate lexical category for a corner case of the language we reuse something that already exists, and name resolution deals with the rest.
(This general approach could grow some specific bugs over the years of course.)

[fmease]

I do understand where you're coming from. However, tuple indices are already blurry from a lexical vs. syntactical perspective due to us splitting float literals as you obviously know. I'm aware though that introspecting tokens like this in the parser is slightly unorthodox (well, just like float and punctuation splitting).

I'll keep this in mind though. Still, I'd like to hear what T-lang thinks assuming crater comes back somewhat okay.

If T-lang and you reject this idea, I'll accept that and I'll just resort to fixing (3):

tuple index shorthands like S { 0 } are legal in patterns but not in expressions

• they were made grammatical in PR RUST-81235 (2021) w/o lang FCP in order to improve diagnostics
• clearly, they should not be syntactically legal

[craterbot]

🚧 Experiment pr-154492 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🎉 Experiment pr-154492 is completed!
📊 7 regressed and 3 fixed (869250 total)
📊 5055 spurious results on the retry-regressed-list.txt, consider a retry¹ if this is a significant amount.
📰 Open the summary report.

⚠️ If you notice any spurious failure please add them to the denylist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

Footnotes

1. re-run the experiment with crates=https://crater-reports.s3.amazonaws.com/pr-154492/retry-regressed-list.txt ↩

[fmease]

@craterbot run mode=check-only crates=https://crater-reports.s3.amazonaws.com/pr-154492/retry-regressed-list.txt

[craterbot]

👌 Experiment pr-154492-1 created and queued.
🤖 Automatically detected try build 53f0b05
⚠️ Try build based on commit 5072669, but latest commit is 9fb994b. Did you forget to make a new try build?
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[rust-bors]

☔ The latest upstream changes (presumably #155720) made this pull request unmergeable. Please resolve the merge conflicts.