Where: assets/profiler-engine.js's parseCSV char-loop (else if (c === '"') { inQuotes = true; } fires on any " encountered while accumulating a field, regardless of position) vs pandas' C parser, which only treats a quote as significant when it's the very first character of a field.
The gap: a common "space after delimiter" export style (Male, "White", 25) has a leading space before the quote in each quoted field. Pandas' parser (no skipinitialspace) sees the leading space as real field content, so the quote isn't at the true start of the field and is preserved literally. The JS engine's state machine doesn't check position - any " flips inQuotes on - so it silently strips the quote characters, changing the parsed value.
Repro: a file space_before_quote.csv:
sex, race, age
Male, "White", 25
Female, "Black", 30
Male, "White", 45
Female, "Asian", 22
>>> import pandas as pd
>>> from faircode import profile
>>> df = pd.read_csv('space_before_quote.csv')
>>> [g['label'] for g in profile(df)['dimensions'][0]['groups']]
[' "White"', ' "Asian"', ' "Black"']Through the JS engine on the identical file:
> E.profile(E.parseCSV(fs.readFileSync('space_before_quote.csv','utf8'))).dimensions
.find(d => d.name === ' race').groups.map(g => g.label)
[ ' White', ' Asian', ' Black' ]Both engines "succeed" - neither errors - but disagree on what the group label literally is (quotes present vs stripped).
Why it matters: this isn't a parse error on either side, so nothing surfaces it - it's a silent divergence in what a protected group's label is. Any downstream consumer that matches labels exactly (a --reference baseline, a --map override, a saved comparison) would silently disagree between the CLI and the web profiler on an ordinary Excel-exported CSV with this common export style.
Suggested fix: only enter quote mode when " is the very first character accumulated for the current field (mirror pandas: only when the field buffer is still empty and no non-quote character has been seen yet), so a quote appearing after other content is treated as a literal character.
Where:
assets/profiler-engine.js'sparseCSVchar-loop (else if (c === '"') { inQuotes = true; }fires on any"encountered while accumulating a field, regardless of position) vs pandas' C parser, which only treats a quote as significant when it's the very first character of a field.The gap: a common "space after delimiter" export style (
Male, "White", 25) has a leading space before the quote in each quoted field. Pandas' parser (noskipinitialspace) sees the leading space as real field content, so the quote isn't at the true start of the field and is preserved literally. The JS engine's state machine doesn't check position - any"flipsinQuoteson - so it silently strips the quote characters, changing the parsed value.Repro: a file
space_before_quote.csv:Through the JS engine on the identical file:
Both engines "succeed" - neither errors - but disagree on what the group label literally is (quotes present vs stripped).
Why it matters: this isn't a parse error on either side, so nothing surfaces it - it's a silent divergence in what a protected group's label is. Any downstream consumer that matches labels exactly (a
--referencebaseline, a--mapoverride, a saved comparison) would silently disagree between the CLI and the web profiler on an ordinary Excel-exported CSV with this common export style.Suggested fix: only enter quote mode when
"is the very first character accumulated for the current field (mirror pandas: only when the field buffer is still empty and no non-quote character has been seen yet), so a quote appearing after other content is treated as a literal character.