Feature/pairwise peptide correlations heatmap by leventetn · Pull Request #39 · UKHD-NP/proteopy

leventetn · 2026-09-28T12:05:01Z

Add `pr.pl.pairwise_peptide_correlations_heatmap`

Purpose

COPF assigns peptides to proteoforms based on their pairwise correlations, but ProteoPy had no way to look at those correlations. This PR adds a per-protein heatmap, so users can check whether the proteoforms COPF reports actually form correlated peptide blocks.

Key changes

New plot: pr.pl.pairwise_peptide_correlations_heatmap(adata, protein, ...) in pl/copf.py
- Draws one protein per call from adata.uns["pairwise_peptide_correlations"] (output of pr.tl.pairwise_peptide_correlations()).
- Clusters rows and columns with the same tree, built from 1 − PCC. method sets the linkage method, linkage accepts a precomputed tree, and cluster=False turns clustering off.
- margin_color adds annotation strips from any .var column(s). The default is proteoform_id from pr.tl.peptide_clusters_from_dendograms(). Strips never share a colour, and each strip gets its own legend, placed clear of all labels.
- Missing data:
  - NaN correlations are drawn hatched, treated as r = 0 when clustering, and reported in a warning.
  - Peptides with no correlations appear as empty rows and columns.
  - Peptides filtered out after the correlations were computed are dropped, with a warning.
- Input checks:
  - Clear errors for a per-batch correlation table, a precomputed linkage of the wrong size, and an invalid margin_color or save.
  - A warning when there are fewer than 3 samples.
- Follows AGENTS.md: labels come from var["peptide_id"], the default ordering rule applies when cluster=False, and print_stats / show / save behave as in other plots.
Refactor (separate commit): the private helper utils.copf.reconstruct_corrs_df_symmetric_from_long_df became utils._matrix_wrangling.reconstruct_symmetric_matrix_from_long.
- It is now generic, raises instead of using assert, and has an allow_missing option. It is not part of the public API.
- In tl/copf.py and test_copro.py, only the rename and black formatting changed; an AST comparison confirms this.
Test fixes: the Karayel test now passes observed=False, and the multi-protein-mapping test no longer uses duplicate .var names.

Tests / lint

New tests: 47 for the heatmap and 10 for the helper.
Full suite: pytest tests/ gives 547 passed.
flake8 is clean; pylint (E/F) scores 10.00/10.

proteopy/utils/copf.py → new proteopy/utils/_matrix_wrangling.py with reconstruct_symmetric_matrix_from_long(df, var_a_col, var_b_col, value_col, *, diagonal=1.0) — generic naming/docstring, diagonal param, and a raise ValueError (with np.isclose) replacing the bare assert. Not exported anywhere (private). Updated the 3 production call sites in tl/copf.py and the test imports; removed the stray test that lived in tests/utils/helpers.py. New tests/utils/test_matrix_wrangling.py — 9 short one-concern tests (basic reconstruction, symmetry, mirroring, diagonal, int-vs-str selectors, generic labels, single pair, missing-pair raise, conflict raise)

pr.pl.pairwise_peptide_correlations_heatmap(adata, protein, ...) in pl/copf.py, registered in pl/init.py. Single protein; reads adata.uns["pairwise_peptide_correlations"], reconstructs the per-protein matrix, draws a sns.clustermap with symmetric row/col clustering (method, unified linkage shadow, cluster toggle) and one-or-more margin_color annotation strips (default cluster_id). Tick labels from var["peptide_id"]; 1e6 NOISE left as an ordinary category; full numpydoc docstring.

New tests/pl/test_pairwise_peptide_correlations_heatmap.py — 16 tests (signature lock, return semantics, symmetry, linkage pass-through, cluster=False, str-vs-list strips, peptide_id labels, validation, save/non-mutation).

…ourhood_union the pandas test now sets observed=False, and the peptide test now represents multiple protein mappings without duplicate variable names

Silently wrong output: A table with several correlations per peptide pair (e.g. …_by_batch) plotted whichever batch came last. It now raises. A precomputed linkage of the wrong size drew a wrong tree. It now raises with the expected shape. Crashes: Missing values in an annotation column: drawn light gray, with an "NA" legend entry. Colour dicts: color_scheme dicts keyed by the column's real values (e.g. {0.0: …}) now work. NaN correlations (missing intensities, constant peptides): cells are drawn hatched with a legend entry, and are treated as r = 0 for clustering, with a warning. NaN no longer blocks the default cluster=True. The helper gained allow_missing=False, which the heatmap sets to True. Peptides filtered out after the correlations were computed: left out with a warning. print_stats without a cluster_id column: it now prints stats per annotation column, in legend order. pdf/svg backends: the plot now works when saved as vector graphics. Layout and readability: One legend per annotation column, placed to the right of all labels. The figure grows if needed, and plt.tight_layout() was removed because it left a gap under the annotation strip. Several annotation strips never share a colour. The title sits on the dendrogram. Peptides that have no correlations at all are shown as empty rows instead of disappearing. Tick labels default to "auto" so labels on large proteins don't overlap. Defaults, validation and AGENTS.md compliance: margin_color defaults to "proteoform_id". With cluster=False, a categorical peptide_id order is respected. Clear errors for an empty or repeated margin_color and for missing COPF columns (with a hint to run pr.tl.peptide_clusters_from_dendograms). An invalid save type is rejected before drawing. A warning when there are fewer than 3 samples. The docstring now has Raises, Warns and Notes sections (data scale, NaN behaviour of the upstream correlation). Tests: heatmap tests went from 16 to 47 and helper tests from 9 to 10. Each new test that targets a fix fails on the earlier version.

read-the-docs-community · 2026-09-28T12:07:24Z

Documentation build overview

📚 proteopy | 🛠️ Build #35035747 | 📁 Comparing 64fc278 against latest (3661567)

🔍 Preview build

6 files changed · + 1 added · ± 5 modified

+ Added

api/generated/proteopy.pl.pairwise_peptide_correlations_heatmap.html

± Modified

idf-io

Please review the comments.

Gist:

Remove params ax and print_stats
When type of margin_color is list, if color_scheme passed, it also must be a list of matching number of elements, which will be used in that order for the different margins and are all passed to _resolve_color...

New asks:

Please add a screenshot of a smoke test in a jupyter notebook to see how the function behaves and that it works. 1-2 plots max

idf-io · 2026-10-06T06:40:59Z


 **Plotting** (`pr.pl`)

+- `pairwise_peptide_correlations_heatmap()`: per-protein clustered


make less verbose: 1 line "- pairwise_peptide_correlations_heatmap(): new feature"

idf-io · 2026-10-06T07:02:22Z

+    yticklabels: bool | str = "auto",
+    figsize: tuple[float, float] = (8.0, 8.0),
+    show: bool = True,
+    ax: bool = False,


I thought we had agreed on the standard: if plotting function coopts figure/axes, return None and dont include ax parameter as it is useless. Please remind me if different.

idf-io · 2026-10-06T07:04:45Z

+    Parameters
+    ----------
+    adata : AnnData
+        :class:`~anndata.AnnData` carrying COPF annotations.


peptide cluster annotations and pairwise peptide correlations output

idf-io · 2026-10-06T07:08:22Z

+        are assigned across all columns together, so strips never share
+        a colour, and each column gets its own legend, placed right of
+        all heatmap labels. Legend entries follow the column's category
+        order if it is categorical, otherwise the lexicographic order of
+        the ``str``-coerced values; store the column as an ordered
+        :class:`pandas.Categorical` to change it. Missing values are
+        drawn in light gray.


All of this is implicit and not relevant for user. Keep docstrings simple to avoid user reading fatigue.*(1)

idf-io · 2026-10-06T07:09:50Z

+        peptides follow the category order of ``adata.var["peptide_id"]``
+        if it is categorical, otherwise lexicographic order. ``NaN``
+        correlations (e.g. from constant peptides) are drawn hatched and
+        treated as ``r = 0`` when building the tree.


idf-io · 2026-10-06T11:34:05Z

+def test_returns_none_when_ax_false():
+    adata = _copf_adata()
+    out = _function()(adata, protein="P1", show=False, ax=False)
+    assert out is None


Test that return is always None

idf-io · 2026-10-06T11:42:41Z

+def test_nan_correlations_cluster_false_draws():
+    adata = _with_nan_correlation(_copf_adata())
+    out = _function()(adata, protein="P1", cluster=False, show=False, ax=True)
+    assert isinstance(out, Axes)


idf-io · 2026-10-06T11:45:04Z

+def test_print_stats_per_margin_color(capsys):
+    adata = _copf_adata()
+    adata.var = adata.var.drop(columns=["cluster_id"])
+    _function()(
+        adata,
+        protein="P1",
+        margin_color="proteoform_id",
+        print_stats=True,
+        show=False,
+    )
+    out = capsys.readouterr().out
+    assert "Peptide correlation summary" in out
+    assert "Per proteoform_id" in out


Not necessary if we remove the param

idf-io · 2026-10-06T11:49:12Z

+def test_print_stats_groups_follow_legend_order(capsys):
+    adata = _copf_adata()
+    adata.var["grp"] = [2, 10, 2, 10, 2, 2, 10, 2]
+    axm = _function()(
+        adata,
+        protein="P1",
+        margin_color="grp",
+        print_stats=True,
+        show=False,
+        ax=True,
+    )
+    legend = [t.get_text() for t in axm.figure.legends[0].get_texts()]
+    table = capsys.readouterr().out.split("Per grp")[1].splitlines()[2:]
+    stats = [line.split()[0] for line in table if line.strip()]
+    assert legend == ["10", "2"]
+    assert stats == legend


idf-io · 2026-10-06T12:05:07Z

+
+
+@pytest.mark.parametrize("cluster", [True, False])
+def test_rendered_cells_match_correlations(cluster):


This is arguably the most important test in the suite. Well done.

- Remove ax and print_stats; always return None. - Support shared and per-strip annotation palettes, resolving both through _resolve_color_scheme. - Preserve shared flat color lists and validate per-strip palette counts. - Omit peptides absent from the correlation table without warning. - Extract input validation into _validate_heatmap_inputs(). - Shorten docstrings and the changelog; correct the default annotation description to proteoform_id. - Replace placeholder examples with a runnable synthetic AnnData example. - Update API and rendering tests; cover both representations of peptides mapping to multiple proteins. Validation: 61 heatmap tests, 129 summarization tests, and 13 doctest statements passed.

leventetn · 2026-10-07T14:31:01Z

Smoke tests:

idf-io · 2026-10-07T21:55:51Z

Could you please increase the padding between the peptide square matrix and the margin annotation bars? Only a little (e.g. 0.5 x row height) but enough to show they are separate things.

Also can you please make the labels of the margin annotations be shown only in the rows? I think this highlights that they are annotations since the common labels between rows and columns are the peptide labels.

In addition, I think the dendograms look weirdly flat from the second level to the leaves. Can you please check it's correct becuase the distance between the top two clusters is so big in comparison to the other distances that it appears like this?

Updated copf.py: - Added half-cell padding between the matrix and annotation bars. - Kept annotation names only on the row bars.

leventetn · 2026-10-08T14:38:03Z

leventetn · 2026-10-09T09:21:33Z

New example for multi annotation margin.

leventetn added 5 commits September 26, 2026 10:53

tests/pairwise_peptide_correlations_heatmap

6f3424d

New tests/pl/test_pairwise_peptide_correlations_heatmap.py — 16 tests (signature lock, return semantics, symmetry, linkage pass-through, cluster=False, str-vs-list strips, peptide_id labels, validation, save/non-mutation).

fixed/test warnings for karayel_2020 and summarize_peptides_by_neighb…

f56302a

…ourhood_union the pandas test now sets observed=False, and the peptide test now represents multiple protein mappings without duplicate variable names

idf-io requested changes Oct 6, 2026

View reviewed changes

leventetn added 2 commits October 8, 2026 00:00

fix: seaborn mathplotlib deprecated warning silenced

072b0a7

fix: margin annotation

753cc05

Updated copf.py: - Added half-cell padding between the matrix and annotation bars. - Kept annotation names only on the row bars.

added pairwise_peptide_correlations_heatmap functionto sphinx api

3db9057

idf-io reviewed Oct 9, 2026

View reviewed changes

Comment thread proteopy/pl/copf.py

fix: added example for pairwise_peptide_correlations_heatmap

64fc278

idf-io self-requested a review October 9, 2026 09:29

idf-io approved these changes Oct 9, 2026

View reviewed changes

idf-io merged commit b336e80 into main Oct 9, 2026
7 checks passed

idf-io deleted the feature/pairwise_peptide_correlations_heatmap branch October 9, 2026 09:35


		Plotting (`pr.pl`)

		- `pairwise_peptide_correlations_heatmap()`: per-protein clustered



		@pytest.mark.parametrize("cluster", [True, False])
		def test_rendered_cells_match_correlations(cluster):

Conversation

leventetn commented Sep 28, 2026

Add pr.pl.pairwise_peptide_correlations_heatmap

Purpose

Key changes

Tests / lint

Uh oh!

read-the-docs-community Bot commented Sep 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation build overview

Uh oh!

idf-io left a comment

Choose a reason for hiding this comment

Gist:

New asks:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

leventetn commented Oct 7, 2026

Uh oh!

idf-io commented Oct 7, 2026

Uh oh!

leventetn commented Oct 8, 2026

Uh oh!

Uh oh!

leventetn commented Oct 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Add `pr.pl.pairwise_peptide_correlations_heatmap`

read-the-docs-community Bot commented Sep 28, 2026 •

edited

Loading