Skip to content

Patch function data type#709

Open
d-chambers wants to merge 5 commits into
masterfrom
patch-function-data-type
Open

Patch function data type#709
d-chambers wants to merge 5 commits into
masterfrom
patch-function-data-type

Conversation

@d-chambers

@d-chambers d-chambers commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Description

This PR updates DASCore's data_type handling for patch functions, following the direction discussed in #704.

patch_function now supports explicit output data_type handling:

Decorator value Behavior
data_type=None Preserve the returned patch's data_type.
data_type="" Clear the returned patch's data_type.
data_type="some_value" Set the returned patch's data_type to that value.

The default remains None to preserve existing behavior for user/downstream code. DASCore patch functions can now opt in to clearing stale labels or setting known derived-product labels.

New/updated function outputs:

Function(s) Output data_type
dft(output="FFT"), stft, spectrogram, rfft fourier_transform
dft(output="AS") amplitude_spectrum
dft(output="PS") power_spectrum
dft(output="PSD") power_spectral_density
fbe frequency_band_energy
stalta stalta
kurtosis kurtosis
envelope envelope
correlate, correlate_shift correlation
tau_p tau_p
dispersion_phase_shift dispersion
phase_weighted_stack phase_weighted_stack
integrate, differentiate, hilbert, normalize, standardize, angle, full, any, all ""

This also normalizes DASCore-assigned data_type values to snake_case, adds them to VALID_DATA_TYPES, updates default colormap keys, and preserves the pre-STFT data_type so istft() round trips do not leave fourier_transform on restored patches.

Closes #704, #511.

Meaning of data_type

data_type is an optional human-readable label for the kind of data contained in a patch. It is not the canonical source of physical meaning.

The canonical physical interpretation should come from:

  • data_units: what physical units the data are in.
  • coords / coordinate units: what axes the data are sampled over.
  • history: how the data were transformed.

data_type should be used as a lightweight descriptor or classifier, useful for display defaults, plotting choices, grouping, and quick user inspection. It should not be used for correctness-critical logic unless the function explicitly documents that requirement, as in existing functions that require data_type="velocity" before converting velocity to strain rate.

The practical policy is:

Situation data_type behavior
Output is still the same measured quantity, just filtered/resampled/selected/reordered Preserve existing data_type.
Output is a known derived product with a stable meaning Set a specific snake_case data_type.
Output changes physical meaning but no stable label is appropriate Clear data_type to "".
Function needs a specific input label Use required_attrs={"data_type": ...}.
Function sets output label through patch_function Use @patch_function(data_type="...").
Function intentionally clears stale labels Use @patch_function(data_type="").
Function should leave whatever it returns untouched Use default @patch_function() / data_type=None.

The guiding principle is: stale or misleading data_type is worse than missing data_type. However, we do not clear data_type by default for patch functions in case this will break existing user code.

Checklist

I have (if applicable):

  • referenced the GitHub issue this PR closes.
  • documented the new feature with docstrings and/or appropriate doc page.
  • included tests. See testing guidelines.
  • added the "ready_for_review" tag once the PR is ready to be reviewed.

Summary by CodeRabbit

  • New Features

    • Added many new data type labels for spectral and signal analyses (Fourier, PSD, amplitude spectrum, envelope, correlation, dispersion, tau_p, stalta, kurtosis, PWS, etc.).
  • Bug Fixes

    • Improved and standardized how transforms set and restore the patch data_type to ensure consistent metadata across processing steps.
  • Chores

    • Canonicalized data_type naming to snake_case and expanded default colormap mappings for new types.
  • Documentation

    • Added notes describing data_type usage and patch attribute handling.
  • Tests

    • New and updated tests validating data_type lengths and decorator-driven data_type behavior.

@coderabbitai

coderabbitai Bot commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@d-chambers, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 14 minutes and 6 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: f931de3b-12c3-44c2-a251-9819e775ad30

📥 Commits

Reviewing files that changed from the base of the PR and between 773e79d and 58b9098.

📒 Files selected for processing (1)
  • docs/notes/patch_attrs.qmd
📝 Walkthrough

Walkthrough

This PR standardizes data type metadata across dascore by extending the patch_function decorator with a data_type parameter, defining canonical lowercase-underscore type strings, updating constants and colormaps, normalizing transform output metadata, and registering all patch functions with their appropriate data types.

Changes

Data type metadata standardization

Layer / File(s) Summary
patch_function decorator data_type parameter
dascore/utils/patch.py
patch_function decorator signature now includes data_type: str | None = None parameter. When a string value is provided, it sets/overwrites the output patch's attrs.data_type; empty string clears it; None leaves it unchanged. Docstring and examples document the behavior. Implementation batches data_type and history updates into a single update_attrs call.
Data type constants and colormaps
dascore/constants.py
VALID_DATA_TYPES gains 12 new canonical type strings including fourier_transform, amplitude_spectrum, power_spectrum, power_spectral_density, frequency_band_energy, stalta, kurtosis, envelope, correlation, tau_p, dispersion, phase_weighted_stack. max_lens["data_type"] increased from 20 to 32. DEFAULT_COLORMAPS keys are converted to underscore-style and expanded with new mappings for all supported data types.
Fourier transform metadata
dascore/transform/fourier.py
DFT_OUTPUT_DATA_TYPE_MAP output type keys change from title-cased to snake_case (amplitude_spectrum, power_spectrum, power_spectral_density). DFT internal data_type changes to fourier_transform. STFT registers with data_type="fourier_transform" and stores original data_type in _pre_stft_data_type attribute for restoration. ISTFT filters _stft* keys and explicitly restores data_type from _pre_stft_data_type.
Transform output metadata normalization
dascore/transform/fbe.py, dascore/transform/kurtosis.py, dascore/transform/stalta.py
FBE sets data_type="frequency_band_energy" in both regular and dB modes. Kurtosis sets data_type="kurtosis" and refines window validation to use direct round() result. STA/LTA sets data_type="stalta".
Apply data_type decorator across modules
dascore/proc/aggregate.py, dascore/proc/basic.py, dascore/proc/correlate.py, dascore/transform/differentiate.py, dascore/transform/dispersion.py, dascore/transform/fft.py, dascore/transform/hilbert.py, dascore/transform/integrate.py, dascore/transform/spectro.py, dascore/transform/taup.py
All patch functions register with explicit data_type. Aggregation (any, all) and general operations (imag, angle, standardize, full, differentiate, integrate, hilbert) use data_type="" to preserve input type. Specialized transforms register with their types: correlate*"correlation", Fourier operations → "fourier_transform", envelope"envelope", phase_weighted_stack"phase_weighted_stack", dispersion_phase_shift"dispersion", tau_p"tau_p".
Test validation
tests/test_core/test_attrs.py, tests/test_transform/test_fbe.py, tests/test_transform/test_fourier.py, tests/test_transform/test_kurtosis.py, tests/test_transform/test_stalta.py, tests/test_utils/test_patch_utils.py
Constants test validates all VALID_DATA_TYPES fit within max_lens. Transform tests verify output data_type matches canonical lowercase-underscore values. New TestPatchFunction tests verify decorator sets/overwrites data_type, preserves when unspecified, clears via empty string, and batches attribute updates into single update_attrs call.

Possibly related PRs

  • DASDAE/dascore#681: Introduces canonical lowercase-underscore data_type names for STALTA/FBE/Kurtosis that this PR extends.
  • DASDAE/dascore#523: Prior work on STFT/ISTFT and Fourier metadata that overlaps the Fourier-related changes here.
  • DASDAE/dascore#693: Earlier Fourier metadata refactor related to DFT output labeling and STFT registration.

Suggested labels

documentation, ready_for_review, proc, patch, transform

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title 'Patch function data type' clearly summarizes the main change—updating the @patch_function decorator to support explicit data_type handling.
Description check ✅ Passed The description provides comprehensive context: problem statement, feature details with tables, rationale, policy, and closes referenced issues. It documents the new decorator semantics and lists affected functions.
Docstring Coverage ✅ Passed Docstring coverage is 95.89% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch patch-function-data-type

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot added documentation Improvements or additions to documentation ready_for_review PR is ready for review proc Related to processing module patch related to Patch class transform Related to transform operations labels Jun 13, 2026
@codecov

codecov Bot commented Jun 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.93%. Comparing base (bab9f84) to head (58b9098).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #709   +/-   ##
=======================================
  Coverage   99.93%   99.93%           
=======================================
  Files         140      140           
  Lines       12171    12179    +8     
=======================================
+ Hits        12163    12171    +8     
  Misses          8        8           
Flag Coverage Δ
unittests 99.93% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

github-actions Bot commented Jun 13, 2026

Copy link
Copy Markdown

✅ Documentation built:
👉 Download
Note: You must be logged in to github and a DASDAE member to access the link.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation patch related to Patch class proc Related to processing module ready_for_review PR is ready for review transform Related to transform operations

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant