Where: faircode/loaders.py's _sniff_delimiter() (used by the CLI/MCP's actual file-reading path via loaders_extra.read_table()) vs assets/profiler-engine.js's sniffDelimiter()/logicalRowDelimiterCounts() (the browser's only reader).
The gap: Issue #439 fixed the CLI's old "sniff only for an unrecognized extension" behavior by making _read_delimited() always sniff via Python's csv.Sniffer, over the full 8192-byte sample. But logicalRowDelimiterCounts() in the JS engine deliberately samples only the first 5 logical (quote-aware) rows before deciding. For a file that is consistently delimited in its first 5 rows but has one messy row later in the sample (e.g. a free-text field with a stray, unquoted delimiter character - a common real-world data-entry artifact), the two engines now disagree: JS confidently locks onto the real delimiter from the first 5 rows; Python's csv.Sniffer scans further into the sample, hits the inconsistency, raises csv.Error, and silently falls back to the extension-implied default (, for .csv). This is a regression the fix for #439 introduced, and tests/test_js_parity.py's one sniffing test only exercises a case where both algorithms already agree.
Repro:
sex;race;age
M;White;25
F;Black;30
M;White;45
F;Asian;22
M;White;50
F;Black; recorded twice;29
M;Asian;33
>>> from faircode.loaders_extra import read_table
>>> df = read_table('minimal_messy.csv')
>>> list(df.columns), df.shape
(['sex;race;age;'], (7, 1)) # collapsed into one bogus column
> const E = require('./assets/profiler-engine.js');
> E.parseCSV(fs.readFileSync('minimal_messy.csv','utf8')).columns
[ 'sex', 'race', 'age' ] // correctly split
Running it end-to-end: faircode profile minimal_messy.csv reports "Rows: 7 Columns: 1 ... Representation score: 100/100 (Grade A)" with a single bogus column, while the JS engine correctly detects Sex/Race/Age as separate demographic dimensions.
Fix direction: Make logicalRowDelimiterCounts() sample more rows (or the same byte window Python uses) so both engines see the same evidence, or make _sniff_delimiter() tolerant of one inconsistent row within a larger sample (a majority/consistency-across-most-rows rule instead of csv.Sniffer's all-or-nothing approach).
Where:
faircode/loaders.py's_sniff_delimiter()(used by the CLI/MCP's actual file-reading path vialoaders_extra.read_table()) vsassets/profiler-engine.js'ssniffDelimiter()/logicalRowDelimiterCounts()(the browser's only reader).The gap: Issue #439 fixed the CLI's old "sniff only for an unrecognized extension" behavior by making
_read_delimited()always sniff via Python'scsv.Sniffer, over the full 8192-byte sample. ButlogicalRowDelimiterCounts()in the JS engine deliberately samples only the first 5 logical (quote-aware) rows before deciding. For a file that is consistently delimited in its first 5 rows but has one messy row later in the sample (e.g. a free-text field with a stray, unquoted delimiter character - a common real-world data-entry artifact), the two engines now disagree: JS confidently locks onto the real delimiter from the first 5 rows; Python'scsv.Snifferscans further into the sample, hits the inconsistency, raisescsv.Error, and silently falls back to the extension-implied default (,for.csv). This is a regression the fix for #439 introduced, andtests/test_js_parity.py's one sniffing test only exercises a case where both algorithms already agree.Repro:
Running it end-to-end:
faircode profile minimal_messy.csvreports "Rows: 7 Columns: 1 ... Representation score: 100/100 (Grade A)" with a single bogus column, while the JS engine correctly detectsSex/Race/Ageas separate demographic dimensions.Fix direction: Make
logicalRowDelimiterCounts()sample more rows (or the same byte window Python uses) so both engines see the same evidence, or make_sniff_delimiter()tolerant of one inconsistent row within a larger sample (a majority/consistency-across-most-rows rule instead ofcsv.Sniffer's all-or-nothing approach).