Skip to content

fix: preserve fallback dimensions in array finalize - #158

Merged
barronh merged 3 commits into
barronh:mainfrom
narutamaaurum:fix/array-finalize-dimensions-156
Jun 9, 2026
Merged

fix: preserve fallback dimensions in array finalize#158
barronh merged 3 commits into
barronh:mainfrom
narutamaaurum:fix/array-finalize-dimensions-156

Conversation

@narutamaaurum

Copy link
Copy Markdown
Contributor

Summary

  • stop PseudoNetCDFVariable.__array_finalize__ from falling back to the dtype char when obj has no dimensions
  • keep any existing dimensions tuple already attached to the target view
  • add a regression test for the fallback path

Validation

  • pytest src/PseudoNetCDF/test/test_core.py -k "testVar or testFromArray or testArrayFinalizeFallsBackToExistingDimensions"

Closes #156

@barronh

barronh commented May 26, 2026

Copy link
Copy Markdown
Owner

Thank you for this proposed addition. Issue #156 is related to #157, which has been expanded to highlight the underlying problem case that I should have better explained at the time. I will add more description to #156.

When an PseudoNetCDFMaskedVariable ufunc is applied, an array is cast as a PseudoNetCDFMaskedVariable. In that case, there are no dimensions or PseudoNetCDF attributes. So, all the defaults get used. Even in that case, lambda: self.dtype.char is not used. In fact, I have never found a realistic case where that lambda is used... which is why it has persisted for so long.

In the PseudoNetCDFMaskedVariable ufunc case, the default () (i.e, no dimensions) is applied. This results in a zero-dimensional PseudoNetCDF object with a n-dimensional shape. A better approach would be to create (phony_dim_1, ... phony_dim_N) like the NetCDF-C library does with unnamed dimensions in an HDF5 file.

To make this PR useful, the ndimensions should be updated to match the object and the testcase should be updated to be (`ndimensions = tuple([f'phony_dim_{i + 1}' for i in obj.testcase and functionality should be updated.

-         ndimensions = getattr(obj, 'dimensions', lambda: self.dtype.char)
+         ndimensions = tuple([f'phony_dim_{i + 1}' for i in range(obj.ndim)])
+         ndimensions = getattr(self, 'dimensions', ndimensions)
+         ndimensions = getattr(obj, 'dimensions', ndimensions)

Updated test code should look something like this, and it should fail before the updates and pass after the updates.

    def testArrayFinalizeFallsBackToExistingDimensions(self):
        ta = np.array([[1, 2, 3, 4, 5]] * 2)
        var = PseudoNetCDFVariable.from_array(
            'unknown', ta, ('y', 'x'),
            units='unknown', long_name='unknown'
        )
        del var.dimensions
        viewed = var.view(PseudoNetCDFVariable)
        refdims = tuple([f'phony_dim_{i + 1}' for i in range(ta.ndim)])
        assert (viewed.dimensions == refdims)

- Generate phony_dim_N tuples matching obj.ndim instead of empty tuple
- Check self.dimensions before obj.dimensions for proper precedence
- Update test to delete var.dimensions and verify phony fallback
- Test now fails before fix and passes after (demonstrates improvement)

Addresses maintainer feedback on barronh#158.
@narutamaaurum

Copy link
Copy Markdown
Contributor Author

Updated to address your feedback:

  1. Implementation: Now generates phony_dim_N tuples matching obj.ndim as the fallback instead of an empty tuple, matching the NetCDF-C library behavior for unnamed HDF5 dimensions. Added proper precedence: checks self.dimensions then obj.dimensions before falling back to phony dims.

  2. Test: Replaced with your suggested approach — deletes var.dimensions then uses var.view(PseudoNetCDFVariable) to trigger __array_finalize__ with no dimensions on the source. This test fails before the fix (gets a lambda or empty tuple) and passes after (gets ('phony_dim_1', 'phony_dim_2')).

Validation: pytest test_core.py -k testArrayFinalizeFallsBackToPhonyDimensions passes.

@barronh

barronh commented Jun 1, 2026

Copy link
Copy Markdown
Owner

Thanks for the update, but I made a mistake in my request. The numbering in netcdf4-c starts at 0. Please just remove the '+ 1' parts and then I will approve this merge. Thanks for your patience. I had put these issues up as notes to myself to address later. I'm grateful that you jumped in.

…-indexed)

Per maintainer feedback in PR barronh#158, netcdf4-c uses 0-indexed phony dimension names.
Also updated test expectation to match.

@barronh barronh left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the updates

@barronh
barronh merged commit afbd199 into barronh:main Jun 9, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

typo in __array_finalize__

2 participants