Skip to content

genClusteredData*: return the ground-truth clusters list ready to pass to css() #161

Description

@gregfaletto

Summary

genClusteredData() / genClusteredDataWeighted() / genClusteredDataWeightedRandom() return list(X, y, Z, mu) but not the ground-truth cluster assignments — even though the block structure they generate (consecutive runs of cluster_size proxy columns) is exactly what css() requires as its clusters argument. Every consumer has to hand-reconstruct the blocks from prose documentation.

Priority

P2 — enhancement (API ergonomics with a correctness angle). A user who miscounts the block boundaries (e.g. assumes weak signals precede proxies) silently gets a wrong clustering — no error, just quietly degraded results. The generators know the truth and should hand it over.

Evidence

  • Return construction at finalizeGenClusteredData.R:42: list(X=X, y=y, Z=Z, mu=mu).
  • Column ordering was verified empirically in review (n = 200,000): exactly [n_clusters × cluster_size proxy block | k_unclustered weak signals | noise].
  • Every existing consumer reconstructs by hand: the advanced-usage vignette (clusters <- list("Z_clust" = 1:10)), the css() example (css.R:131: clusters <- list(cluster1 = 1:4)), the getCssPreds example, and the README.

Proposed fix (sketch)

Add a clusters element to the returned list in finalizeGenClusteredData (index.Rmd ~line 4156) — backward compatible (pure addition):

clusters <- split(seq_len(n_clusters * cluster_size),
                  rep(seq_len(n_clusters), each = cluster_size))
names(clusters) <- paste0("c", seq_len(n_clusters))   # match css()'s auto-naming style
list(X = X, y = y, Z = Z, mu = mu, clusters = clusters)

Then:

  • document the new element in all three generators' @return;
  • simplify the examples/vignettes/README to css(X = data$X, y = data$y, clusters = data$clusters, ...) — which also serves as living documentation of the column layout.

Handle the sig_clusters = 0 / n_clusters = 0-adjacent cases consistently (empty list) — note open #141 (allow n_clusters = 0) would want clusters = list() there too.

Tests to add: returned clusters matches the documented block layout for all three generators (including a weighted variant), and feeding it straight into css() reproduces the hand-constructed-clusters result exactly.


Found in the 2026-07-21 whole-package review (column-layout verification executed at scale against 674af7e). Fixes belong in index.Rmd (litr source) — cssr/ is generated (see build.R).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions