Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,25 @@

## [2.0.0] - Unreleased

### JOSS reviewer polish (2026-08-16)

- Make `saxsabs estimate-k --ref` optional so the CLI can use the same built-in
NIST SRM 3600 curve as the Python API; reject `--ref-q-col`/`--ref-i-col`
without `--ref`.
- Gate CLI `estimate-k` and `subtract-buffer` (and the `minimal_2d` example)
through the intensity-state contract; refuse unlabeled intensity before $K$.
- Add `estimate-k --thickness-cm` (centimetres; Workbench Tab 1 remains mm) and
a `subtract-buffer` subcommand.
- Stamp `1/cm` only when writers receive `absolute_cm^-1` plus an explicit
cm$^{-1}$ unit; readers expose `i_abs`/`i_rel` only for those assessed states.
A unitless metadata label `absolute` is ambiguous, not cm$^{-1}$.
- Point README and paper at the unreleased 2.0.0 tree on `main`, not GitHub
Release v1.1.1. Record canSAS XSD / punx checks as offline 2026-08-15, not CI.
- Tighten the paper's K-estimator, interface-equivalence, and `minimal_2d`
claims (9×9 homemade radial average, not pyFAI; unknown uncertainty; not the
BL19B2 campaign runner), and remove stale Draft-PR #1 merge instructions from
the README and readiness notes.

### Submission-candidate hardening (2026-08-12)

- Require finite positive inherited `thickness_cm` and a non-empty
Expand Down
6 changes: 4 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ git clone https://github.com/D-sudoasd/SASAbs.git
cd SASAbs
python -m pip install -e ".[dev]"
pytest -q
ruff check src tests
ruff check SASAbs.py saxs_mpl_style.py src tests paper/*.py scripts/*.py
```

Install `.[gui]`, `.[hdf5]`, `.[io]`, or `.[bl19b2]` only when the change needs
Expand All @@ -33,7 +33,9 @@ Keep pull requests focused. For a behavior change:
- add or update focused tests;
- keep reusable scientific logic in `src/saxsabs/` and GUI orchestration separate;
- update public CLI/API documentation when its behavior changes;
- run `pytest -q` and `ruff check src tests` locally;
- run `pytest -q` and
`ruff check SASAbs.py saxs_mpl_style.py src tests paper/*.py scripts/*.py`
locally;
- describe the workflow, validation performed, and any remaining limitations.

Maintainers review pull requests for scientific input semantics, provenance,
Expand Down
41 changes: 27 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,12 @@
reusable data writers, and provenance checks. The result and the processing
record remain reviewable together.

Reviewers should use the unreleased 2.0.0 tree on
[`main`](https://github.com/D-sudoasd/SASAbs). GitHub Release
[v1.1.1](https://github.com/D-sudoasd/SASAbs/releases/tag/v1.1.1) is an earlier
archive and is not this candidate. Do not treat the Zenodo concept DOI as a
version DOI for 2.0.0.

<p align="center">
<a href="#quick-start"><strong>Quick start</strong></a> ·
<a href="#choose-a-workflow">Choose a workflow</a> ·
Expand Down Expand Up @@ -77,7 +83,7 @@ a display server.

| Route | Best for | Start here |
| --- | --- | --- |
| **CLI utilities** | normalization, header and 1D parsing, robust K estimation | `saxsabs --help` |
| **CLI utilities** | normalization, header and 1D parsing, gated K estimation, gated buffer subtraction | `saxsabs --help` |
| **SAXSAbs Workbench** | interactive K calibration, batch processing, external-1D scaling | `saxsabs-workbench --lang en` |
| **Strict BL19B2 runner** | validated campaign inputs under current BL19B2 conventions | [batch runbook](docs/bl19b2_abs2d_batch_runbook.md) |
| **Python API** | reusable scientific calculations and file I/O | [API reference](docs/api.md) |
Expand Down Expand Up @@ -116,25 +122,30 @@ not experimental evidence.

## Reproducible example

The bundled example generates deterministic synthetic dark, background,
standard, and sample frames, then runs the package reduction APIs:
The bundled example plants deterministic synthetic dark, background,
standard, and sample frames on a 9×9 array, subtracts a NIST blank in detector
space, and reduces with a homemade integer-bin radial average (not pyFAI):

```bash
python examples/minimal_2d/run_minimal_2d_pipeline.py
```

It writes inspectable CSV, TSV, and XML outputs, plus HDF5 when `h5py` is
installed. The acceptance summary requires `k_relative_error < 0.005` and
`sample_max_relative_error < 0.01`. See the
installed. The script gates the standard profile as `relative` before $K$,
writes `absolute_cm^-1` metadata, and checks that the XML exposes `i_abs`
rather than `i_rel`. The acceptance summary requires `k_relative_error < 0.005`
and `sample_max_relative_error < 0.01`. See the
[example documentation](examples/minimal_2d/README.md) for construction details
and expected files.

<p align="center">
<img src="assets/readme/kfactor-demo.png" width="92%" alt="Deterministic synthetic K-factor example showing retained and rejected ratios.">
</p>

> This example checks software arithmetic and generated file content. It is not
> measured-beamline validation or independent third-party format validation.
> This example recovers a planted synthetic $K$ and sample curve from a 9×9
> homemade radial average and checks labeled file content. It is not pyFAI
> integration, BL19B2 campaign validation, measured-beamline validation, or
> independent third-party format validation.

## Documentation

Expand All @@ -151,10 +162,12 @@ and expected files.

Absolute calibration depends on a suitable reference, detector geometry,
monitor semantics, transmission, thickness, and instrument-specific provenance.
The strict 2D workflow currently targets BL19B2 conventions. canSAS1d output is
checked against the official version 1.1 XSD. NXcanSAS output passes project-local
round-trip tests and punx 0.3.5 with its bundled v2018.5 definitions; current
NeXus definitions and third-party consumers have not yet been verified.
The strict 2D workflow currently targets BL19B2 conventions. canSAS1d and
NXcanSAS layouts are covered by project-local round-trip tests. An offline
check on 15 August 2026 validated the deterministic example against the official
canSAS1d 1.1 XSD and punx 0.3.5 with its bundled v2018.5 definitions; that check
is not in CI. Current NeXus definitions and third-party consumers have not been
verified.

## Development

Expand All @@ -176,9 +189,9 @@ python scripts/check_submission_readiness.py \
```

Run the strict command from the exact branch and commit that will be submitted.
If Draft PR #1 is merged first, check out the resulting clean `main`, update
`submitted_branch` and `submitted_commit` in the confirmation JSON, and rerun
the gate. A PASS recorded before the merge does not cover the merge commit.
PR #1 is already on `main`. A PASS recorded on an earlier revision does not
cover a later commit; update `submitted_branch` and `submitted_commit` and rerun
the gate on the revision sent to JOSS.

After the strict local gate passes, verify the same commit, branch, visible
README and paper, repository identity, and successful CI run against GitHub:
Expand Down
58 changes: 48 additions & 10 deletions SASAbs.py
Original file line number Diff line number Diff line change
Expand Up @@ -1340,10 +1340,34 @@ def should_skip_all_existing(existing_flags, policy):
write_nxcansas_h5 = None

try:
from saxsabs.io.parsers import read_external_1d_profile as _core_read_external_1d_profile
from saxsabs.io.parsers import (
profile_intensity as _core_profile_intensity,
profile_uncertainty as _core_profile_uncertainty,
read_external_1d_profile as _core_read_external_1d_profile,
)
except Exception:
_core_profile_intensity = None
_core_profile_uncertainty = None
_core_read_external_1d_profile = None


def _profile_intensity(profile):
if _core_profile_intensity is not None:
return _core_profile_intensity(profile)
for key in ("intensity", "i_abs", "i_rel"):
if key in profile:
return np.asarray(profile[key], dtype=np.float64)
raise KeyError("profile has no intensity array")


def _profile_uncertainty(profile):
if _core_profile_uncertainty is not None:
return _core_profile_uncertainty(profile)
for key in ("uncertainty", "err_abs", "err_rel"):
if key in profile:
return np.asarray(profile[key], dtype=np.float64)
raise KeyError("profile has no uncertainty array")

try:
from saxsabs.io.calibrated2d import (
SCHEMA_VERSION as CALIBRATED2D_SCHEMA_VERSION,
Expand Down Expand Up @@ -5657,18 +5681,29 @@ def subtract_external_absolute_buffer(

if subtract_buffer is None:
raise RuntimeError("formal buffer subtraction kernel is unavailable")
sample_profile = {
"intensity_state": "absolute_cm^-1",
"intensity_unit": "1/cm",
"i_col": "I_abs_cm^-1",
"operator_provenance": {
"intensity_state": "absolute_cm^-1",
"corrections_applied": '["k","thickness"]',
},
}
args = (
np.asarray(sample_q, dtype=np.float64),
np.asarray(sample_i, dtype=np.float64),
np.asarray(sample_err, dtype=np.float64),
np.asarray(buffer_profile["x"], dtype=np.float64),
np.asarray(buffer_profile["i_rel"], dtype=np.float64),
np.asarray(buffer_profile["err_rel"], dtype=np.float64),
_profile_intensity(buffer_profile),
_profile_uncertainty(buffer_profile),
)
result = subtract_buffer(
*args,
alpha=alpha,
alpha_uncertainty=alpha_uncertainty,
sample_profile=sample_profile,
buffer_profile=buffer_profile,
)
if result.err_statistical is None:
raise RuntimeError("buffer kernel did not return statistical uncertainty")
Expand Down Expand Up @@ -6134,8 +6169,11 @@ def align_profile_to_x(self, x_target, ref_profile, name):

xr, yr, er = self._regularize_xy_triplet(
ref_profile["x"],
ref_profile["i_rel"],
ref_profile.get("err_rel"),
_profile_intensity(ref_profile),
ref_profile.get(
"uncertainty",
ref_profile.get("err_abs", ref_profile.get("err_rel")),
),
min_points=2,
name=name,
)
Expand Down Expand Up @@ -6713,8 +6751,8 @@ def run_external_1d_batch(self):
else:
thk_cm_used = fixed_thk_cm
thickness_source = "tab3_fixed_thickness_input"
i_abs = np.asarray(prof["i_rel"], dtype=np.float64) * scale_factor
err_abs = np.asarray(prof["err_rel"], dtype=np.float64) * abs(scale_factor)
i_abs = _profile_intensity(prof) * scale_factor
err_abs = _profile_uncertainty(prof) * abs(scale_factor)
else:
sp = self.resolve_external_sample_params(fp, meta_map, monitor_mode)
norm_s = sp["norm"]
Expand Down Expand Up @@ -6742,8 +6780,8 @@ def run_external_1d_batch(self):
thickness_source = str(provenance["thickness_source"])
scale_factor = k

s_i = np.asarray(prof["i_rel"], dtype=np.float64)
s_e = np.asarray(prof["err_rel"], dtype=np.float64)
s_i = _profile_intensity(prof)
s_e = _profile_uncertainty(prof)
x = np.asarray(prof["x"], dtype=np.float64)

self.assert_external_profile_axis_compatible(prof, bg_prof, "BG")
Expand Down Expand Up @@ -10867,7 +10905,7 @@ def _get_std_reference_data(self):
from saxsabs.io.parsers import read_external_1d_profile
prof = read_external_1d_profile(ref_path)
q_user = prof["x"]
i_user = prof["i_rel"]
i_user = _profile_intensity(prof)
return get_reference_data(key, q_user=q_user, i_user=i_user)
elif key == "Water_20C":
temp_c = self.t1_water_temp.get()
Expand Down
25 changes: 15 additions & 10 deletions SUBMISSION_READINESS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,18 @@

Updated: 16 August 2026 (Asia/Shanghai)

Review the unreleased 2.0.0 tree on `main`, not GitHub Release v1.1.1. Do not
create `v2.0.0`, a GitHub Release, or a Zenodo version archive during review.

## Locally verified

- Full source suite: PASS in a fully provisioned Python 3.13 environment; exact
count and duration are retained in the dated external validation record.
- Ruff: root modules, package, tests, paper scripts, and submission gate pass.
- README: 5 local images and all local links resolve; SVG/image audit passes.
- Minimal 2D example: K and sample maximum relative errors are
`0.001933697...`; CSV, TSV, XML, and HDF5 outputs are written.
- Minimal 2D example: 9×9 homemade radial average (not pyFAI) recovers planted
K and sample maximum relative errors of `0.001933697...`; CSV, TSV, XML, and
HDF5 outputs are written with unknown uncertainty.
- Fresh-copy distribution build: wheel and sdist PASS from a source tree
outside every Git checkout. The exact archive inventory is retained in the
dated external validation record; the sdist includes README assets,
Expand All @@ -30,10 +34,11 @@ Updated: 16 August 2026 (Asia/Shanghai)
have green push and Draft-PR runs for the complete matrix. Immutable commit
IDs and run URLs belong in the dated external validation record rather than
this tracked file, because editing the evidence here creates a new HEAD.
- External format checks: the deterministic example's canSAS1d XML validates
against the official 1.1 XSD with zero errors. Its NXcanSAS HDF5 output passes
punx 0.3.5 with bundled v2018.5 definitions (97 OK, 0 WARN, 0 ERROR); current
NeXus definitions and third-party consumers remain unverified.
- External format checks (offline 15 August 2026, not in CI): the deterministic
example's canSAS1d XML validated against the official 1.1 XSD with zero
errors. Its NXcanSAS HDF5 output passed punx 0.3.5 with bundled v2018.5
definitions (97 OK, 0 WARN, 0 ERROR); current NeXus definitions and
third-party consumers remain unverified.

## Must be resolved before submission

Expand All @@ -59,10 +64,10 @@ python scripts/check_submission_readiness.py \
--manual-confirmations path/to/submission-confirmations.json
```

The gate must run on the exact branch and commit submitted to JOSS. If the
Draft PR is merged first, rerun it on the clean resulting `main` commit and
record `submitted_branch` and `submitted_commit` accordingly; pre-merge
evidence is not evidence for a later merge commit.
The gate must run on the exact branch and commit submitted to JOSS. PR #1 is
already on `main`; rerun the gate on the clean `main` commit that will be
submitted and record `submitted_branch` and `submitted_commit` accordingly.
Evidence from an earlier revision is not evidence for a later commit.

After that local PASS, run:

Expand Down
Loading