Quantify a design with a numbering gap, with a warning in the log and results.txt, instead of refusing it

[trishorts]

Project of origin: QuantProject (consumed by sdrf, PXReprise, aging)

Draft, waiting on smith-chem-wisc/mzLib#1426 and smith-chem-wisc/mzLib#1422. It stays a draft on this fork until smith-chem-wisc#1426 merges, and is opened on smith-chem-wisc/MetaMorpheus once an mzLib release carries both. That release's pin bump is then added to this PR. It is not here yet, so CI here builds against 1.0.594, where TestSdrfDesignKeepsStudyWideReplicatesAndSaysSo, TestSdrfDesignWritesAFractionGapAndBothSidesSaySo and TestSearchQuantifiesADesignWithABiorepGap are expected to fail. Every result below is from a local package of 1.0.594 plus smith-chem-wisc#1422 and smith-chem-wisc#1426 at 2af2d4f8a (9.9.1247).

What

A gap in an experimental design's numbering is now quantified with a warning instead of refused. This covers a condition's biological replicates, a biorep's fractions, a fraction's technical replicates, and, for TMT, a plex's fractions and technical replicates. The warning goes to the log and into results.txt. A duplicate (two files at one condition, biorep, fraction and techrep, or at one fraction and techrep of a plex) is still refused.

Before, a label-free design with any gap was refused (Condition "X" biorep N is missing!), and the search skipped quantification with one warning. The TMT design window refused fraction and techrep gaps. The TMT parser never checked them, so a run quantified them silently.

Numbers are never closed up:

• A biorep number can name a subject across conditions. Control 1–3 and treated 4–6 is a common SDRF numbering, and replicate 4 of one condition can be replicate 4 of another (matched tissue, paired designs).
• A gap can be a lost sample or file. Closing it up hides that.

One rule, here and in mzLib smith-chem-wisc#1426

smith-chem-wisc#1426 makes mzLib's SdrfLabelFreeDesign (--sdrfDesign) keep the SDRF's numbers. It refuses the same thing this PR refuses (a duplicate) and notes the same gaps in the same words:

	mzLib smith-chem-wisc#1426, SDRF report	this PR, run warning
biorep gap	Condition 'FA': biological replicates 3, 4, kept as the SDRF numbers them. ...	Condition "FA": biological replicates 3, 4, quantified as numbered; biological replicates 1, 2 are not in the design. ...
fraction gap	Condition 'A' biorep 1: fractions 1, 3, kept as the SDRF numbers them. ...	Condition "A" biorep 1: fractions 1, 3, quantified as numbered; fraction 2 is not in the design. ...
techrep gap	Condition 'A' biorep 1 fraction 1: technical replicates 1, 3, kept as ...	Condition "A" biorep 1 fraction 1: technical replicates 1, 3, quantified as numbered; ...
duplicate	refused	refused

TestSdrfDesignWritesAFractionGapAndBothSidesSaySo runs one fraction-gap SDRF through both sides and checks both messages. Each repository keeps its own quoting: mzLib quotes a condition with ', and MetaMorpheus's existing design messages use ".

Behaviour

in the design	before	now
numbers run 1..N	quantified	quantified (unchanged)
label-free: a gap in bioreps, fractions or techreps, or numbering not starting at 1	refused; the search skips quantification	quantified as numbered, one warning per gap, in the log and results.txt
label-free: duplicate	refused	refused (unchanged)
TMT window: a gap in a plex's fractions or techreps	refused on Save	saved, gaps listed in the Save and Validate messages
TMT run: a gap	quantified, nothing said	quantified, one warning per gap, in the log and results.txt
TMT: no plex, a number below 1, duplicate in a plex	refused by the window	refused (unchanged)

• Label-free (ExperimentalDesign):
  • GetErrorsInExperimentalDesign now refuses only duplicates. The message is unchanged.
  • New GetWarningsInExperimentalDesign returns one line per gap.
• TMT (TmtExperimentalDesign, Replicate-structure validation is implemented twice, and the untestable copy is the one with the extra rules smith-chem-wisc/MetaMorpheus#2793): the window's private ValidateDesign rules move into EngineLayer as GetErrorsInDesign and GetWarningsInDesign, so a test can reach them. The window calls them. This answers Replicate-structure validation is implemented twice, and the untestable copy is the one with the extra rules smith-chem-wisc/MetaMorpheus#2793's "only the untestable copy has the rules". Its third copy, AnnotatePlexWindow.ValidateAll, is not moved.
• Where the warning goes:
  • the search task (label-free and TMT) and the glyco task: Warn, plus AddTaskSummaryText, so it is in results.txt (as SkipPredictedSpectralAngles already does);
  • the command line's design check before a run (Program.ResolveExperimentalDesign, both design types), when reporting to the console;
  • the GUI: the label-free design window after Save; the TMT window in its Save and Validate messages.
• --sdrfDesign help text said replicates are renumbered within each condition. It now says they are copied as the SDRF numbers them, and that added numbers and gaps are printed.

Calibration and averaging (CalibrationTask, SpectralAveragingTask) copy the design under the new file names, carrying the numbers through unchanged. Before, a design with a gap was an error there too: they warned and wrote no new design, so the search after them would quantify without one (read from the code, not run). Now they write it.

Fractions carry meaning, and a gap keeps it. FlashLFQ's match-between-runs transfers only between fractions at most one apart (FlashLfqEngine.cs:832). Fractions numbered 1 and 3 therefore exchange no transfers, which is what the numbers say (read, not run).

Why this needs mzLib smith-chem-wisc#1422 in the same release

FlashLFQ's normalization and Bayesian step walked replicates 0..max. smith-chem-wisc#1422 makes them walk the numbers that exist. Measured against 1.0.594 (no smith-chem-wisc#1422):

• Biorep gap: TestSearchQuantifiesADesignWithABiorepGap runs, but biorep normalization silently does nothing. The four files keep their 2 : 3 : 4 : 5 intensity ratio (max/min 2.5), because the first condition has no biorep 1, which 1.0.594 uses as the reference. With Unit Tests for Serialization and Deserialization Possibly Needed smith-chem-wisc/MetaMorpheus#1422 the four intensities agree within 1%.
• Fraction and techrep gaps (TestSearchQuantifiesFractionAndTechrepGapsAsNumbered) and TMT (TmtFractionGap_IsQuantifiedAsNumberedAndWarned) pass on 1.0.594 as well. These fixtures do not build the cases Unit Tests for Serialization and Deserialization Possibly Needed smith-chem-wisc/MetaMorpheus#1422 fixes in fraction normalization (a lost biorep across fractions), so they show no dependency, not its absence.

smith-chem-wisc#1422's description lists the other failures: Sequence contains no elements in fraction normalization, and KeyNotFoundException in the Bayesian step (smith-chem-wisc#2886).

Tested

New:

• TestBiorepGapIsAWarningNotAnError:
  • bioreps 1, 2, 4 → no error, exactly one warning, naming 3;
  • control 1, 2 / treated 3, 4 → one warning, for treated;
  • 1..N → none;
  • a fraction gap and a techrep gap → warnings naming the biorep and fraction;
  • a duplicate → still refused.
• TestSearchQuantifiesADesignWithABiorepGap: a search with normalization on, A = 2, 3 and B = 4, 5.
  • Quantification is not skipped.
  • Both warnings are in the log and in results.txt.
  • The protein table's columns are Intensity_A_2, _A_3, _B_4, _B_5.
  • The four normalized peptide intensities agree within 1%.
• TestSearchQuantifiesFractionAndTechrepGapsAsNumbered: the same five files numbered with fraction and techrep gaps (1, 3), and without them. Both are quantified with normalization on, the gaps are warned about in the log and results.txt, and every file's normalized intensity is identical in the two.
• TmtDesignErrorsAreTheWindowRules and TmtDesignGapsAreWarningsNotErrors:
  • the window's refusals hold: no plex, below 1, duplicate in a plex;
  • two plexes may each have (1, 1);
  • gaps give exactly the expected warnings, each plex judged on its own numbers.
• TmtFractionGap_IsQuantifiedAsNumberedAndWarned: two files of one plex as fractions 1, 3 and as 1, 2. The gap is warned about in the log and results.txt, and the peptide and protein quantification tables are identical.
• TestSdrfDesignWritesAFractionGapAndBothSidesSaySo: see above.

Changed:

• TestExperimentalDesignErrors: its non-consecutive biorep, fraction and techrep cases now expect no error and one warning each. Its duplicate, missing-file and non-integer cases are unchanged.
• TestSdrfDesignRenumbersStudyWideReplicatesAndSaysSo is now TestSdrfDesignKeepsStudyWideReplicatesAndSaysSo: treated 3, 4 is written as 3, 4, read back with no error, and the pre-run check exits 0 and prints the warning.
• TestSdrfDesignRefusesAFractionGapAndWritesNothing is now TestSdrfDesignRefusesADuplicateAndWritesNothing. A fraction gap is no longer refused, so the refusal it pins is now the one that is left.
• GlycoTestWithBadExperimentalDesignFile: its bad design was a techrep gap, which is now valid. It is now a duplicate, so the test still checks that a bad design skips quantification and the search runs.

Falsified: each of ten mutations turned tests red, on the final code, and the restored source passed again (116 passed, 0 failed):

• biorep refusal put back (4 red);
• fraction-gap refusal put back (4);
• label-free duplicate not refused (3);
• no label-free warnings at all (6);
• no search-task warning (2);
• no command-line warning (2);
• no results.txt line (2);
• no TMT warning at search (1);
• TMT duplicate not refused (1);
• no TMT gap warnings (2).

Not covered: the glyco task's warning lines and both GUI windows' messages have no test (Test.csproj does not reference GUI).

Against master's code: with smith-chem-wisc#1426 and smith-chem-wisc#1422, master's TestSdrfDesignRenumbersStudyWideReplicatesAndSaysSo fails (it expects renumbered 3 -> 1, 4 -> 2). That is why smith-chem-wisc#1426 is labelled Breaks Integration Tests.

Full Test project against 9.9.1247: 2918 passed, 0 failed, 1 skipped (fd4742c15).

Not in this PR

• The FlashLFQ app still throws on a biorep, fraction or techrep gap on its master. It is the third copy of this rule. The same change, with the same warning text, is drafted as A numbering gap in the experimental design is quantified with a warning; only a duplicate is refused FlashLFQ#3, which waits on the mzLib release carrying Unit Tests for Serialization and Deserialization Possibly Needed smith-chem-wisc/MetaMorpheus#1422.
• Isobaric quantification does not combine fractions. Each fraction file gets its own channel columns (Sample1_TmtFracA_126, Sample1_TmtFracB_126 for one sample), and the fraction number is not read. That is unchanged here, and why a TMT fraction gap changes no number.
• mzLib's SdrfIsobaricDesign never refused a gap, so it already follows the rule, but its report does not note one.

🤖 Generated with Claude Code