fix(core): one home for the number-display policy — and percent stops drifting between cell and measure (#4576)

.changeset/one-number-display-home-4576.md

@@ -0,0 +1,16 @@
+---
+'@object-ui/core': minor
+'@object-ui/i18n': minor
+---
+
+One home for the number-display policy — and a percent stops meaning two different things between a list cell and a dashboard measure
+
+`formatDisplayNumber`, `shouldGroupDisplayNumber` and `DisplayNumberFormatOptions` move from `@object-ui/i18n` into `@object-ui/core`. `@object-ui/i18n` re-exports all three under the same names, so every existing import path keeps working unchanged and both spellings resolve to the same function object; nothing published was removed.
+
+The move is what fixes the bug. `@object-ui/core`'s `formatMeasure` needed exactly this policy and could not import it — `core` is the React-free engine and is a runtime dependency of React-free consumers (the `object-ui` VS Code extension, `@object-ui/data-objectstack`), while `i18n` depends on `i18next`/`react-i18next` and peer-depends on React. So `formatMeasure` carried a parallel `Intl` implementation, recorded at both ends as deliberate duplication, and the two drifted in the one place a hand-built string and `Intl` disagree. A German session read `1.234,5 %` from a list cell and `1.234,5%` from a dashboard measure showing the same number. The function is pure, so the boundary was never a property of the code — only of where the code sat; moving it down removes the obstacle instead of working around it. `core` imports nothing from `i18n`, so the new edge adds no cycle.
+
+**Behaviour change — a measure's percent sign now follows the locale.** `formatMeasure` appended a literal `%` in every locale; it now renders the locale's own percent convention, the same one the list-cell `formatPercent` has used since the fix to its own machine-locale defect. Measured to change output in de, fr, es, ru, sv, cs, fi (a no-break space appears before the sign), tr (the sign moves to the FRONT: `%1.234,5`) and ar (its own percent sign plus U+061C). English, Japanese and Chinese are byte-identical — their convention is a bare trailing sign — which is why this was invisible in an English session.
+
+**No numeral moves, in any locale, at any magnitude.** The obvious route to the locale's convention is `Intl`'s `style: 'percent'`, but that style expects a fraction, so a value already in percentage points would have to be divided by 100 for `Intl` to multiply it straight back — and that round trip is lossy. Measured, it moves 27,581 of 1,200,013 ordinary-magnitude en-US forms at rounding ties (`0.175` at two decimals becomes `0.17%` instead of `0.18%`), plus `MAX_SAFE_INTEGER` and everything from 1e23 up, where `100,000,000,000,000,000,000,000%` becomes `99,999,999,999,999,990,000,000%`. The percentage points are formatted directly instead, through a new `style: 'percentPoints'` on `DisplayNumberFormatOptions`; that route was measured to produce a byte-identical percent affix to `style: 'percent'` across all 171 locale tags tested while moving none of those 1,200,013 forms. Callers holding a fraction keep using `style: 'percent'`, whose behaviour is unchanged — naming the two cases apart is what stops the next caller from reaching for the lossy one.
+
+`@object-ui/i18n`'s entry declaration is byte-identical, but the declaration it points at now lives in `@object-ui/core` and the package gains that dependency, so it takes the same minor bump rather than a patch.

packages/app-shell/src/views/metadata-admin/inspectors/DatasetDefaultInspector.measureLocale.test.tsx

@@ -127,7 +127,15 @@ describe('dataset inspector format sample follows the display locale (objectui#4
     );
     // The decimal COMMA is the point: `12.3%` and `12,3%` read as different
     // numbers to the two audiences, not as the same one restyled.
-    expect(sampleText()).toBe('12,3%');
+    //
+    // PIN MOVED (objectui#4576). The German expectation gained the no-break
+    // space before the sign. `formatMeasure` appended a literal '%' in every
+    // locale; it now renders the locale's own percent convention, the one a
+    // list cell has used since #4553. The en sample in the case below is
+    // UNMOVED, which is what says this was a convention change and not a
+    // numeral one. Same class as the two moves in
+    // `packages/core/src/utils/__tests__/dataset-format.locale.test.ts`.
+    expect(sampleText()).toBe('12,3\u00a0%');
   });
 
   it('keeps the en percent sample byte-identical (must-not-change)', () => {

packages/core/src/index.ts

@@ -59,6 +59,12 @@ export * from './utils/dashboard-filters.js';
 export * from './utils/merge-filters.js';
 export * from './utils/compare-to.js';
 export * from './utils/chart-series.js';
+// The ONE number-display formatter (objectui#4033) — grouping policy, display
+// locale and the percent convention. It lived in `@object-ui/i18n` until
+// objectui#4576; it is pure, and living above `core` was what kept
+// `dataset-format` below from reaching it (so the two drifted). `@object-ui/i18n`
+// re-exports these names unchanged, so both import paths name the same symbol.
+export * from './utils/number-display.js';
 export * from './utils/dataset-format.js';
 // Pivot lookup-key encoders, shared by every cross-tab renderer so the
 // dashboard widget and the report renderer key their buckets identically

packages/core/src/utils/__tests__/dataset-format.locale.test.ts

@@ -67,12 +67,22 @@ describe('formatMeasure follows the display locale (objectui#4566)', () => {
   it('formats a percent measure in the threaded locale', () => {
     // The decimal COMMA is the whole point: `60.8%` and `60,8%` read as
     // different numbers to the two audiences, not as the same one restyled.
-    expect(formatMeasure(0.608_333_333, '0.0%', undefined, undefined, 'de-DE')).toBe('60,8%');
+    //
+    // PIN MOVED (objectui#4576). The German expectation gained the no-break
+    // space before the sign. That is not this case's subject drifting — it is
+    // the divergence #4566 recorded and declined to fix here, now closed: a
+    // German list cell has written `60,8${NBSP}%` since #4553, and the measure
+    // appended a literal '%'. The en-US line beside it is UNMOVED, which is
+    // what says this was a convention change and not a numeral one.
+    expect(formatMeasure(0.608_333_333, '0.0%', undefined, undefined, 'de-DE')).toBe(`60,8${NBSP}%`);
     expect(formatMeasure(0.608_333_333, '0.0%', undefined, undefined, 'en-US')).toBe('60.8%');
   });
 
   it('honours a declared percentScale in the threaded locale', () => {
-    expect(formatMeasure(1, '0.0%', undefined, 'fraction', 'de-DE')).toBe('100,0%');
+    // PIN MOVED (objectui#4576) — same reason as the case above; the declared
+    // `percentScale` behaviour it actually pins is untouched, and en-US is
+    // byte-identical.
+    expect(formatMeasure(1, '0.0%', undefined, 'fraction', 'de-DE')).toBe(`100,0${NBSP}%`);
     expect(formatMeasure(1, '0.0%', undefined, 'fraction', 'en-US')).toBe('100.0%');
   });
 

packages/core/src/utils/__tests__/dataset-format.percent-convention.test.ts

@@ -0,0 +1,192 @@
+/**
+ * ObjectUI
+ * Copyright (c) 2024-present ObjectStack Inc.
+ *
+ * This source code is licensed under the MIT license found in the
+ * LICENSE file in the root directory of this source tree.
+ */
+
+/**
+ * objectui#4576 — a percent renders as `1.234,5 %` in a list cell and
+ * `1.234,5%` as a dashboard measure. Two conventions for one number.
+ *
+ * `formatPercent` (`@object-ui/fields`) has gone through `Intl` since #4553, so
+ * it gets the locale's percent CONVENTION — German writes a no-break space
+ * before the sign. `formatMeasure` appended a LITERAL '%' to a decimal body, so
+ * it got no space in any locale. The SCALING already agreed (both go through
+ * `percentDisplayValue`); the CONVENTION did not, which is exactly what
+ * `percentDisplayValue`'s doc comment promises can never happen.
+ *
+ * ── PREDICTIONS, written before the run ──
+ * Reverse verification takes the fix out (`git diff` to a patch +
+ * `git checkout --`, never `git stash`) and keeps these expectations.
+ *
+ * RED before the fix (the convention move):
+ *   - every de-DE / fr-FR / ru-RU / es-ES case — they gain U+00A0 before '%'
+ *   - the tr-TR case — the sign moves to the FRONT of the number
+ *   - the ar-EG case — its own percent sign (U+066A) plus U+061C, not ASCII '%'
+ *   - the cross-surface agreement cases, which is the card's actual complaint
+ *
+ * GREEN on BOTH sides (must-not-change pins, each labelled in place):
+ *   - every en-US case, ordinary magnitude AND extreme — this is the discriminator
+ *   - ja-JP / zh-CN, whose percent convention has no space either
+ *   - the rounding-tie cases, which pin the route NOT taken (see below)
+ *
+ * Every case pins at least one locale whose convention has a space AND en-US,
+ * because a lone de assertion is not falsifiable on a German runner. Runner
+ * measured: node v22.22.2, ICU 78.2, machine locale en-US.
+ *
+ * ⚠️ The en cases are not decoration. The obvious way to adopt the locale's
+ * convention is `Intl`'s `style: 'percent'` — the way `formatPercent` does it —
+ * but that style wants a FRACTION, so a value already in percentage points must
+ * be divided by 100 for `Intl` to multiply it straight back, and that round trip
+ * is lossy at rounding TIES. Measured on this runner: 27,581 of 1,200,013
+ * ordinary-magnitude en-US forms move that way (`0.175` at 2 decimals goes from
+ * `0.18%` to `0.17%`), plus `MAX_SAFE_INTEGER` and everything from 1e23 up. The
+ * implementation formats the percentage points DIRECTLY with the percent unit
+ * instead, which was measured byte-identical on the affix across all 171 locale
+ * tags tested and moves none of those 1,200,013 forms. The tie and
+ * extreme-magnitude cases below are what keep that route honest: they go red if
+ * anyone "simplifies" this into the divide-by-100 form.
+ */
+
+import { describe, it, expect } from 'vitest';
+import { formatMeasure } from '../dataset-format';
+
+// The locale's own percent affix, taken from `Intl` rather than hardcoded —
+// this is the SAME source `formatPercent` renders through, so comparing against
+// it is a structural pin on "both surfaces use the locale's convention" rather
+// than a restatement of today's CLDR data.
+function percentAffix(locale: string): { prefix: string; suffix: string } {
+  let prefix = '';
+  let suffix = '';
+  let seenNumber = false;
+  for (const part of new Intl.NumberFormat(locale, { style: 'percent' }).formatToParts(1234.5)) {
+    if (part.type === 'integer' || part.type === 'group' || part.type === 'decimal' || part.type === 'fraction') {
+      seenNumber = true;
+      continue;
+    }
+    if (part.type === 'minusSign' || part.type === 'plusSign') continue;
+    if (seenNumber) suffix += part.value;
+    else prefix += part.value;
+  }
+  return { prefix, suffix };
+}
+
+describe('formatMeasure percent — the locale\'s convention, not a literal sign (#4576)', () => {
+  it('German puts a no-break space before the sign, as the list cell already did', () => {
+    // U+00A0 written as an escape, never as a raw byte.
+    expect(formatMeasure(0.608_333_333, '0.0%', undefined, undefined, 'de-DE')).toBe('60,8\u00a0%');
+    expect(formatMeasure(1, '0.0%', undefined, 'fraction', 'de-DE')).toBe('100,0\u00a0%');
+    expect(formatMeasure(80, '0.0%', undefined, 'whole', 'de-DE')).toBe('80,0\u00a0%');
+    // en-US pinned alongside so the case is falsifiable on a German runner.
+    expect(formatMeasure(0.608_333_333, '0.0%', undefined, undefined, 'en-US')).toBe('60.8%');
+  });
+
+  it('French and Russian do the same; Spanish too', () => {
+    // fr-FR groups with U+202F (narrow no-break space) and spaces the sign with U+00A0.
+    expect(formatMeasure(1234.5, '0.0%', undefined, 'whole', 'fr-FR')).toBe('1\u202f234,5\u00a0%');
+    expect(formatMeasure(1234.5, '0.0%', undefined, 'whole', 'ru-RU')).toBe('1\u00a0234,5\u00a0%');
+    expect(formatMeasure(1234.5, '0.0%', undefined, 'whole', 'es-ES')).toBe('1234,5\u00a0%');
+    expect(formatMeasure(1234.5, '0.0%', undefined, 'whole', 'en-US')).toBe('1,234.5%');
+  });
+
+  it('Turkish puts the sign in FRONT — a convention a literal suffix can never reach', () => {
+    // The strongest case in the file: no amount of "append '%'" produces this,
+    // so it cannot pass by accident.
+    expect(formatMeasure(1234.5, '0.0%', undefined, 'whole', 'tr-TR')).toBe('%1.234,5');
+  });
+
+  it('Arabic uses its OWN percent sign, not ASCII "%"', () => {
+    const out = formatMeasure(50, '0%', undefined, 'whole', 'ar-EG');
+    // U+066A ARABIC PERCENT SIGN, then U+061C ARABIC LETTER MARK.
+    expect(out).toContain('\u066a');
+    expect(out).not.toContain('%');
+  });
+
+  it('agrees with the locale\'s percent affix — the same one the list cell renders through', () => {
+    for (const locale of ['de-DE', 'fr-FR', 'tr-TR', 'ar-EG', 'en-US', 'ja-JP', 'zh-CN']) {
+      const { prefix, suffix } = percentAffix(locale);
+      const out = formatMeasure(1234.5, '0.0%', undefined, 'whole', locale);
+      expect(out.startsWith(prefix), `${locale} prefix`).toBe(true);
+      expect(out.endsWith(suffix), `${locale} suffix`).toBe(true);
+    }
+  });
+
+  // ── must-not-change pins ────────────────────────────────────────────────
+  // GREEN on both sides of the fix. They are the discriminator: this card
+  // changes a rendering CONVENTION, and English has no space in its convention,
+  // so English must not move at all.
+
+  it('MUST NOT CHANGE: ordinary-magnitude en-US is byte-identical', () => {
+    expect(formatMeasure(50, '0%')).toBe('50%');
+    expect(formatMeasure(0.75, '0%')).toBe('75%');
+    expect(formatMeasure(0.608_333_333, '0.0%')).toBe('60.8%');
+    expect(formatMeasure(0, '0%')).toBe('0%');
+    expect(formatMeasure(1, '0%')).toBe('1%');
+    expect(formatMeasure(1, '0.0%', undefined, 'fraction')).toBe('100.0%');
+    expect(formatMeasure(0.6667, '0.0%', undefined, 'fraction')).toBe('66.7%');
+    expect(formatMeasure(0, '0.0%', undefined, 'fraction')).toBe('0.0%');
+    expect(formatMeasure(1, '0.0%', undefined, 'whole')).toBe('1.0%');
+    expect(formatMeasure(0.5, '0.0%', undefined, 'whole')).toBe('0.5%');
+    expect(formatMeasure(80, '0.0%', undefined, 'whole')).toBe('80.0%');
+    expect(formatMeasure(1234.5, '0.0%', undefined, 'whole', 'en-US')).toBe('1,234.5%');
+  });
+
+  it('MUST NOT CHANGE: ja-JP and zh-CN have no space in their convention either', () => {
+    expect(formatMeasure(1234.5, '0.0%', undefined, 'whole', 'ja-JP')).toBe('1,234.5%');
+    expect(formatMeasure(1234.5, '0.0%', undefined, 'whole', 'zh-CN')).toBe('1,234.5%');
+  });
+
+  it('MUST NOT CHANGE: rounding ties keep the AUTHORED decimal (the /100 route is refused)', () => {
+    // 0.175 percentage points at 2 decimals. Formatted directly this is
+    // `0.18%`. Divided by 100 for `Intl`'s `style: 'percent'` to multiply back,
+    // the double lands below the tie and it becomes `0.17%`. This case is the
+    // pin on which route is taken — it goes red the moment someone rewrites
+    // this path as `style: 'percent'` on `display / 100`.
+    expect(formatMeasure(0.175, '0.00%', undefined, 'whole')).toBe('0.18%');
+    expect(formatMeasure(0.305, '0.00%', undefined, 'whole')).toBe('0.31%');
+    expect(formatMeasure(0.35, '0.0%', undefined, 'whole')).toBe('0.4%');
+  });
+
+  it('a tie keeps its digits AND gains the German space \u2014 both halves at once', () => {
+    // Deliberately NOT filed under "MUST NOT CHANGE": the German form moves
+    // (it gains U+00A0), so this case is RED before the fix. Its job is to show
+    // the two properties are independent \u2014 adopting the convention did not cost
+    // the digits, which is the whole argument for the route taken.
+    expect(formatMeasure(0.175, '0.00%', undefined, 'whole', 'de-DE')).toBe('0,18\u00a0%');
+    expect(formatMeasure(0.175, '0.00%', undefined, 'whole', 'en-US')).toBe('0.18%');
+  });
+
+  it('MUST NOT CHANGE: extreme magnitudes keep every digit', () => {
+    // The `/100` round trip corrupts both of these (measured:
+    // `9,007,199,254,740,990%` and `99,999,999,999,999,990,000,000%`).
+    // #4577 measured 24 of its 32,760 combinations moving that way and declined
+    // the swap for it; this implementation does not move them at all.
+    expect(formatMeasure(Number.MAX_SAFE_INTEGER, '0%', undefined, 'whole')).toBe(
+      '9,007,199,254,740,991%',
+    );
+    expect(formatMeasure(1e21, '0%', undefined, 'fraction')).toBe(
+      '100,000,000,000,000,000,000,000%',
+    );
+  });
+
+  it('MUST NOT CHANGE: non-percent formats gain no sign, and the legacy "$" literal still leads', () => {
+    expect(formatMeasure(1234.5, '0,0')).toBe('1,235');
+    expect(formatMeasure(1234.5, '$0.0')).toBe('$1,234.5');
+    expect(formatMeasure(1234.5, '$0.0%', undefined, 'whole')).toBe('$1,234.5%');
+  });
+
+  it('the legacy "$" literal still leads a German percent too \u2014 it sits OUTSIDE the affix', () => {
+    // RED before the fix (the German form gains U+00A0). Pinned separately from
+    // the must-not-change case above so that label stays honest.
+    expect(formatMeasure(1234.5, '$0.0%', undefined, 'whole', 'de-DE')).toBe('$1.234,5\u00a0%');
+  });
+
+  it('MUST NOT CHANGE: a malformed locale tag degrades instead of throwing', () => {
+    // `en_US` (underscore) is the likeliest tenant-config typo and throws from a
+    // bare `Intl.NumberFormat`. The retry lives in `formatDisplayNumber`.
+    expect(() => formatMeasure(50, '0%', undefined, 'whole', 'en_US')).not.toThrow();
+    expect(formatMeasure(50, '0%', undefined, 'whole', 'en_US')).toMatch(/50.?%|%.?50/u);
+  });
+});