From 4bfb6eb2b37fafd9d75664f515b35d788fe842d8 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 14:25:01 +0000 Subject: [PATCH 1/2] docs: add a vignette on reading provenance$coverage (#72) vignettes/coverage-counters.Rmd explains the three per-year counters that cog_geographic_rollup() and cog_peer_compare() attach as provenance$coverage, using the definitions in .coverage_table(), R/rollup.R and R/peers.R: n_units_expected (the ids passed, or the peers), n_units_collected (of those, how many have any row in the spending long view that year) and n_units_reporting (of those, how many have a row in the requested category). It splits the named governments into three groups, defines the collection rate and category participation, and shows that reporting / expected is their product. The evaluated chunks run against the bundled fixture and say so: Georgia cities and Police across FY2011, FY2012, FY2019 and FY2020, a peer comparison, and the census, consistent and per_capita effects. The FY2022 example from the reference pages needs the published corpus and is not evaluated, following population-denominators.Rmd. The article is added to the pkgdown Concepts index. Closes #72 Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Qbe6u2h83wgZdTgJgE4Xh5 --- _pkgdown.yml | 1 + vignettes/coverage-counters.Rmd | 270 ++++++++++++++++++++++++++++++++ 2 files changed, 271 insertions(+) create mode 100644 vignettes/coverage-counters.Rmd diff --git a/_pkgdown.yml b/_pkgdown.yml index 057bc65..6dc5c74 100644 --- a/_pkgdown.yml +++ b/_pkgdown.yml @@ -39,3 +39,4 @@ articles: contents: - total-spending - population-denominators + - coverage-counters diff --git a/vignettes/coverage-counters.Rmd b/vignettes/coverage-counters.Rmd new file mode 100644 index 0000000..734f4b4 --- /dev/null +++ b/vignettes/coverage-counters.Rmd @@ -0,0 +1,270 @@ +--- +title: "Coverage counters: expected, collected, reporting" +output: rmarkdown::html_vignette +vignette: > + %\VignetteIndexEntry{Coverage counters: expected, collected, reporting} + %\VignetteEngine{knitr::rmarkdown} + %\VignetteEncoding{UTF-8} +--- + +```{r setup, include = FALSE} +knitr::opts_chunk$set(collapse = TRUE, comment = "#>") +``` + +# What `provenance$coverage` holds + +`cog_geographic_rollup()` and `cog_peer_compare()` attach a coverage table to their +provenance at `provenance$coverage`. It has one row per requested year and five +columns: `year`, `n_units_collected`, `n_units_reporting`, `n_units_expected` and +`is_census_year`. A second field, `provenance$coverage_mode`, records the `coverage` +argument you passed. + +The table is built from the years you requested. A year in which none of your +governments has a row still gets a row in the table, with `n_units_reporting` equal +to zero. + +Point `USCOGDATA_URL` at the bundled fixture corpus to run the examples without +network access or credentials. The fixture carries FY2011, FY2012, FY2019 and FY2020 +for all 50 states. Every count computed from it is fixture output and not a real +Georgia figure. + +```{r} +library(uscogdata) + +# Point at the bundled offline fixture. In real use, USCOGDATA_URL is set to the +# published corpus URL instead -- see README.md. +Sys.setenv(USCOGDATA_URL = paste0( + system.file("extdata/fixture_corpus", package = "uscogdata"), "/" +)) +``` + +# What each counter counts + +Each counter answers a different question about the governments you named. + +`n_units_expected` counts the governments you asked about. For +`cog_geographic_rollup()` it is the number of distinct ids across every layer of +`govids`. For `cog_peer_compare()` it is the number of peers. The target is the +subject of the comparison and is not counted. + +`n_units_collected` counts how many of those governments have at least one spending +row in the corpus that year, in any category. The verb looks this up in the long view +behind `cog_spending()`, which holds every non-aggregate, non-intergovernmental +expenditure row. A government that Census did not collect from that year has no rows +there, whatever category you asked about. + +`n_units_reporting` counts how many of those governments have at least one row in +the result for that year. The result holds only the category you requested, so this +counter depends on the category. + +A government with a row in the result also has a row in the long view, so in every +year `n_units_reporting <= n_units_collected <= n_units_expected`. + +# Three groups of governments + +The counters split the governments you named into three groups each year: + +1. `n_units_expected - n_units_collected` governments have no spending rows in the + corpus that year. The Census of Governments collects from every government only + in years ending in 2 or 7 and from a sample in other years, so in a sample year + this group includes every government outside that year's sample. +2. `n_units_collected - n_units_reporting` governments were collected that year but + have no rows in your category. A city that contracts its policing to the county + sheriff lands here for `category = "Police"`. Its Police spending is a real zero. +3. `n_units_reporting` governments have rows in your category. + +A government in the first group and a government in the second group look the same in +the result: neither has a row. Only `n_units_collected` tells them apart. + +# Georgia cities and Police spending + +Build the list of Georgia cities with `cog_gov_search()` and roll their Police +spending up for all four fixture years: + +```{r} +ga_cities <- cog_gov_search(state = "GA", type = "city") +nrow(ga_cities) + +police <- cog_geographic_rollup( + govids = list(city = ga_cities$canonical_govid), + category = "Police", + years = c(2011, 2012, 2019, 2020) +) + +coverage <- attr(police, "provenance")$coverage +coverage +``` + +On the fixture, `n_units_expected` is 566 in every year, because it counts the ids +you passed. FY2012 is a census year. Census collected 534 of the 566 cities, and 398 +of those have Police rows. That leaves 32 cities in the first group and 136 in the +second. In the three sample years, Census collected 60 or 103 cities, and nearly all +of them have Police rows. + +You can rebuild `n_units_collected` from the public verbs. The `"direct"` concept +covers every non-intergovernmental expenditure subtype, so asking `cog_spending()` +for every Direct category returns a row for each collected government: + +```{r, message = FALSE} +direct <- cog_spending( + ga_cities$canonical_govid, + years = c(2011, 2012, 2019, 2020), + category = "All Categories", + expenditure_concept = "direct" +) +collected <- unique(direct[, c("year", "canonical_govid")]) +table(collected$year) +``` + +The counts match the `n_units_collected` column above. + +# Two ratios + +Divide the counters to get two rates: + +```{r} +coverage$collection_rate <- coverage$n_units_collected / coverage$n_units_expected +coverage$participation <- coverage$n_units_reporting / coverage$n_units_collected +coverage[, c("year", "is_census_year", "collection_rate", "participation")] +``` + +The collection rate, `n_units_collected / n_units_expected`, is the share of the +governments you named that Census collected that year. It depends on the survey +calendar and the sample, and it is the same for every category. On the fixture it is +0.94 in FY2012 and between 0.11 and 0.18 in the sample years. + +Category participation, `n_units_reporting / n_units_collected`, is the share of the +collected governments that have rows in your category. On the fixture, 0.75 of the +cities collected in FY2012 have Police rows. + +Do not read `n_units_reporting / n_units_expected` as a response rate. It equals the +collection rate times category participation, so a low value can come from a sample +year, from a category many governments do not fund, or from both. For FY2012 it is +398 / 566, about 0.70, and the counters show that most of the shortfall is the 136 +collected cities with no Police rows. + +# Comparing a census year with a sample year + +Participation is higher in the fixture's sample years than in FY2012: 1.00 in FY2011 +and 0.93 in FY2019 and FY2020. The cities Census collects in a sample year are a +different set from the full universe. Join the collected cities to the population +estimate that `cog_gov_search()` returns: + +```{r} +collected <- merge(collected, ga_cities[, c("canonical_govid", "population_acs")]) +tapply(collected$population_acs, collected$year, median, na.rm = TRUE) +``` + +On the fixture, the median collected city has about 1,500 residents in FY2012 and +about 17,500 in FY2019 and FY2020. The sample years collect much larger cities than +the typical Georgia city, and a larger share of them report Police spending. A +sample-year participation rate describes the cities Census sampled that year, so use +a census year to estimate the share of all Georgia cities with Police spending. + +The collection rate shows which kind of year you are looking at. When it is low, treat +both the amounts and the participation rate as statements about the sampled +governments. + +# The FY2022 example on the published corpus + +The reference pages for `cog_geographic_rollup()` and `cog_peer_compare()` describe +the same query for FY2022, a census year. The fixture does not carry FY2022, so the +query needs the published corpus: + +```{r, eval = FALSE} +# Not run: needs USCOGDATA_URL set to the published corpus. The fixture has no FY2022. +ga_cities <- cog_gov_search(state = "GA", type = "city") +police_2022 <- cog_geographic_rollup( + govids = list(city = ga_cities$canonical_govid), + category = "Police", + years = 2022 +) +attr(police_2022, "provenance")$coverage +``` + +Those pages report 393 of 567 Georgia cities with Police rows in FY2022 and attribute +most of the 174-city gap to cities that contract policing to the county sheriff. Run +the chunk and read `n_units_collected` to split the 174 into the two groups: the +cities Census did not collect, and the collected cities with no Police rows. + +# Peer comparisons count peers only + +In `cog_peer_compare()`, `n_units_expected` is the number of peers, and +`n_units_reporting` counts peer rows only. The target is left out of all three +counters. Compare Fitzgerald, Georgia, with the 12 Georgia cities closest to it in +FY2012 population: + +```{r} +fitzgerald <- "132017204209" +peers <- cog_find_peers(fitzgerald, year = 2012, same_state = TRUE, max_peers = 12) +nrow(peers) + +cmp <- cog_peer_compare(fitzgerald, peers, category = "Police", years = c(2012, 2019)) +attr(cmp, "provenance")$coverage +``` + +All 12 peers were collected and report Police spending in FY2012. In FY2019, Census +collected four of them and three have Police rows. Eight peers fall in the first group +and one in the second. The `summary_p25`, `summary_p50` and `summary_p75` rows for +FY2019 are computed from those three peers' rows. + +# What else changes the counters + +`coverage = "census"` drops the sample years before the query runs, so the table has +one row per requested census year: + +```{r} +census_only <- cog_geographic_rollup( + govids = list(city = ga_cities$canonical_govid), + category = "Police", + years = c(2011, 2012, 2019, 2020), + coverage = "census" +) +attr(census_only, "provenance")$coverage +``` + +`coverage = "consistent"` keeps only the governments with rows in every requested +year. The filter runs before the counters are computed, so `n_units_reporting` is the +same in every year. `n_units_expected` and `n_units_collected` are not filtered: + +```{r} +balanced <- cog_geographic_rollup( + govids = list(city = ga_cities$canonical_govid), + category = "Police", + years = c(2012, 2019), + coverage = "consistent" +) +attr(balanced, "provenance")$coverage +``` + +On the fixture, 93 Georgia cities have Police rows in both FY2012 and FY2019. Under +`"consistent"`, `n_units_reporting / n_units_collected` no longer measures category +participation, because the numerator is filtered and the denominator is not. + +`cog_peer_compare()` exempts the target from the `"consistent"` filter, and its +`summary_*` quantiles are computed after the filter. + +With `per_capita = TRUE`, `cog_geographic_rollup()` drops rows whose government has +no population for that year before it counts `n_units_reporting`. Such a government +still counts as expected and collected. `cog_peer_compare()` does not drop these +rows. + +# `is_census_year` + +`is_census_year` is `TRUE` for years ending in 2 or 7. It describes the survey +calendar. Read `n_units_collected` to see how complete a year is: in FY2012, a census +year, the fixture has spending rows for 534 of its 566 Georgia cities. + +# Summary + +- `n_units_expected` is the number of governments you named, or the number of peers + in `cog_peer_compare()`. +- `n_units_collected` is how many of them have any spending row that year. +- `n_units_reporting` is how many of them have a row in your category that year. +- `n_units_collected / n_units_expected` is the collection rate. + `n_units_reporting / n_units_collected` is category participation among collected + governments. +- `n_units_reporting / n_units_expected` mixes the two, so read the counters + separately before you explain a gap. +- A sample year collects a different set of governments from a census year, so + compare participation rates across those years with care. From e3cb9d508db2c0ecc1aa7e4ce7bee03a5d4aaddf Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 15:50:53 +0000 Subject: [PATCH 2/2] docs: ground the census-vs-sample advice in the published corpus (#72) Checked against the published corpus (build of 2026-10-26) through the public API: - FY2022 Georgia cities, Police: 567 expected, 535 collected, 393 reporting. The 174-city gap is 32 not collected and 142 collected with no Police rows. The FY2022 section now states the split. - All US cities: Police participation among collected cities is 0.70, 0.73 and 0.72 in FY2012, FY2017 and FY2022, and 0.93 to 0.95 in every sample year FY2013-FY2023. Among the 3,867 cities collected in both FY2022 and FY2021 it is 0.929 and 0.935; Fire, Libraries and Housing & Community move by 0.01 or less on the same cities. A census year and a sample year collect different sets of governments, so the census-vs-sample section now recommends comparing census years with each other, or the same governments across years, and adds a fixture chunk that measures participation over the 99 Georgia cities collected in both FY2012 and FY2019 (0.97 and 0.94). The fixture caveat now says what data-raw/regenerate_fixture_corpus.R does: each fixture year is a full copy of the published partition. Its FY2012 and FY2019 Georgia Police counts match the published corpus; the newer crosswalk lists 567 Georgia cities where the fixture lists 566. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Qbe6u2h83wgZdTgJgE4Xh5 --- vignettes/coverage-counters.Rmd | 57 ++++++++++++++++++++++++++------- 1 file changed, 45 insertions(+), 12 deletions(-) diff --git a/vignettes/coverage-counters.Rmd b/vignettes/coverage-counters.Rmd index 734f4b4..338739e 100644 --- a/vignettes/coverage-counters.Rmd +++ b/vignettes/coverage-counters.Rmd @@ -24,9 +24,11 @@ governments has a row still gets a row in the table, with `n_units_reporting` eq to zero. Point `USCOGDATA_URL` at the bundled fixture corpus to run the examples without -network access or credentials. The fixture carries FY2011, FY2012, FY2019 and FY2020 -for all 50 states. Every count computed from it is fixture output and not a real -Georgia figure. +network access or credentials. The fixture copies four full years of the published +corpus, FY2011, FY2012, FY2019 and FY2020, from its 2026-08-03 build. Its FY2012 and +FY2019 counts for Georgia cities and Police match the published corpus as of the +2026-10-26 build, except that the newer crosswalk lists 567 Georgia cities where the +fixture lists 566. ```{r} library(uscogdata) @@ -161,9 +163,38 @@ the typical Georgia city, and a larger share of them report Police spending. A sample-year participation rate describes the cities Census sampled that year, so use a census year to estimate the share of all Georgia cities with Police spending. -The collection rate shows which kind of year you are looking at. When it is low, treat -both the amounts and the participation rate as statements about the sampled -governments. +To compare a census year with a sample year, hold the governments fixed. Take the +cities collected in both years and measure participation among them: + +```{r} +both <- intersect( + collected$canonical_govid[collected$year == 2012], + collected$canonical_govid[collected$year == 2019] +) +length(both) + +reported <- unique(police[, c("year", "canonical_govid")]) +sapply(c(FY2012 = 2012, FY2019 = 2019), function(y) { + mean(both %in% reported$canonical_govid[reported$year == y]) +}) +``` + +On the fixture, 99 Georgia cities were collected in both FY2012 and FY2019. Among +them, participation is 0.97 in FY2012 and 0.94 in FY2019, against 0.75 and 0.93 among +all collected cities. Once the cities are the same, the FY2019 rate is no longer +higher than the FY2012 rate. + +The published corpus shows the same pattern for every US city. As of its 2026-10-26 +build, Police participation among collected cities is 0.70, 0.73 and 0.72 in the +census years FY2012, FY2017 and FY2022, and between 0.93 and 0.95 in every sample year +from FY2013 to FY2023. Among the 3,867 cities collected in both FY2022 and FY2021, it +is 0.929 in FY2022 and 0.935 in FY2021. Fire, Libraries and Housing & Community behave +the same way: participation among the same cities moves by 0.01 or less between those +two years, while participation among all collected cities rises by 0.16 to 0.24. + +Compare participation between two census years, or between the same governments in +two years. All collected governments in a census year and all collected governments +in a sample year are two different sets of cities. # The FY2022 example on the published corpus @@ -182,10 +213,11 @@ police_2022 <- cog_geographic_rollup( attr(police_2022, "provenance")$coverage ``` -Those pages report 393 of 567 Georgia cities with Police rows in FY2022 and attribute -most of the 174-city gap to cities that contract policing to the county sheriff. Run -the chunk and read `n_units_collected` to split the 174 into the two groups: the -cities Census did not collect, and the collected cities with no Police rows. +On the published corpus, as of its 2026-10-26 build, the chunk returns 567 expected, +535 collected and 393 reporting. The 174 cities without Police rows split into 32 +that Census did not collect and 142 that were collected and have no Police rows. The +reference pages attribute most of the gap to cities that contract policing to the +county sheriff, and the counters place 142 of the 174 in the second group. # Peer comparisons count peers only @@ -266,5 +298,6 @@ year, the fixture has spending rows for 534 of its 566 Georgia cities. governments. - `n_units_reporting / n_units_expected` mixes the two, so read the counters separately before you explain a gap. -- A sample year collects a different set of governments from a census year, so - compare participation rates across those years with care. +- A sample year collects a different set of governments from a census year. + Compare participation between census years, or between the same governments + across years.