Skip to content

sparse.COO support for harmonize_missing_values and infer_feature_types #305

Description

@sueoglu

ehrdata can store a numeric .X/layer as sparse.COO tensor, but two core functions don't handle it correctly::

  • infer_feature_types crashes on sparse.COO input:
>>> ed.infer_feature_types(edata)  # edata.X is sparse.COO
NotImplementedError: The `order` parameter is not supported

It tries to reshape the sparse array the same way it reshapes a dense one, which sparse.COO.reshape doesn't support. Since several downstream functions (e.g .ep.pp.scale_norm) call infer_feature_types automatically whenever edata.var["feature_type"] isn't set yet, this blocks normal usage of any sparse.COO-backed EHRData

  • harmonize_missing_values silently does nothing for sparse.COO:
    For a sparse array, the implicit 0 fill value is ambiguous, it can mean "not measured" or "measured as zero", but harmonize_missing_values currently treats all numeric layers (dense or sparse) as a no-op, so there's no way to mark genuinely-missing sparse entries as NaN.

TODO

  • infer_feature_types should work on a sparse.COO X/layer without densifying the whole array
  • harmonize_missing_values treats a sparse.COO array's implicit 0 as missing by default, without densifying, with an opt-out mechanism for columns whose 0 is a real, measured value rather than a missing one
  • No behavior change for other types

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions