[jknowles]

Gitea issue #75

What changed

New export: cog_suggestions(govid, years, category, flow = c("spending", "revenue"), concept = NULL, years_present) (R/suggestions.R)

It returns exactly the list that cog_spending() / cog_revenue() attach as attr(x, "provenance")$suggestions, given the years the caller's rows cover. Internally it is a thin wrapper over .build_suggestions() that makes the same call .verb_spendrev() makes, so callers no longer need the seven internal names listed in the issue.

• flow selects the verb. Its view base, subtype column and flow prefixes live in a small .SUGGESTION_FLOWS table next to the function.
• concept is expenditure_concept for spending or revenue_concept for revenue. NULL resolves to the verb's own default ("primary" / "general"). It only changes the result for category = "All Categories", where it sets the candidate subtype scope.
• basis is resolved exactly as .verb_spendrev() resolves an unset basis. On a schema_version < 5 corpus that means "raw", and the result is list(), matching the verbs.
• It does not print the suggestion message the verbs emit.
• Validation runs before .ensure_session() and uses cli errors. Classes match the verbs where one exists (uscogdata_invalid_expenditure_concept, uscogdata_invalid_revenue_concept, uscogdata_all_categories_not_combinable). uscogdata_invalid_flow is new.

Stable order (separate commit, d74fb62). .query_candidate_recipes() used SELECT DISTINCT with no ORDER BY, and .build_suggestions() emits suggestions in candidate order. The query now ends in ORDER BY recipe_id. The sort is inside .build_suggestions()'s path, so the verbs' provenance$suggestions order changes too, and I ran the full suite (results below).

Also updated: NAMESPACE, man/cog_suggestions.Rd (generated with roxygen2 7.3.3, matching RoxygenNote; no other .Rd changed), NEWS.md under a new # uscogdata (development version) heading, and _pkgdown.yml (reference index, "Corpus metadata" section, after cog_recipes).

Choices to review

These follow the signature sketched in the issue, but each is a call I made:

1. category = NULL is refused with an explanatory error. The verbs build no suggestions without a category, so a NULL here can only return list().
2. The cohort is govid only, as a character vector or a data frame with canonical_govid. The verbs' state/type predicate cohort is not exposed, because the issue's signature does not include it.
3. years_present is required. integer(0) means "no rows". NULL is refused, so a forgotten argument cannot read as "no rows in any year". Years outside years are ignored.
4. The verbs are unchanged apart from the sort. .SUGGESTION_FLOWS restates the flow prefixes that cog_spending()/cog_revenue() pass to .verb_spendrev(). The equivalence tests would catch drift between the two, but they are two copies of the same values.
5. DESCRIPTION is still Version: 0.4.0. NEWS.md has a development heading, but I left any 0.4.0.9000 bump to you.

How it was tested

Locally, on Ubuntu 24.04 with R 4.3.3, duckdb 2.0.0 (compiled from CRAN source) and testthat 3.2.1.

Local deviation: the sandbox could not reach extensions.duckdb.org, so cog_open()'s INSTALL httpfs; LOAD httpfs; failed in every session. I ran the suite on a scratch copy of the tree with that one line removed. The fixture is a local path and no test references httpfs. The patch is not in any commit, and CI runs the unpatched code.

• Full suite on this branch: 1,162 expectations, 0 failed, 0 warnings, 0 errors, 2 skipped. The skips are the two opt-in test-live-corpus.R tests.

• Same suite on main (ba57f9c): 1,103 expectations, 0 failed, same 2 skips. The difference is the 59 new expectations in tests/testthat/test-suggestions.R.

• The new tests check that cog_suggestions() returns a list identical() to the verb's for:

  • an empty_year fire (with ig_recipe_id)
  • a suppressed_component fire
  • a three-government cohort
  • a data-frame govid
  • cog_revenue() under both concepts
  • "All Categories" under primary, direct and revenue total
  • expenditure_concept = "total" given Direct-leg years
  • an empty result
  • a doctored schema_version 4 corpus

  They also cover silence, repeated-call identity, sorted order, and every validation error.

• The order tests fail on main's code: two failures. On the fixture, cog_spending("010000226085", 2011, category = "All Categories") lists corrections_other_capital_combined before corrections_combined without the sort.

• pkgdown::check_pkgdown(): no problems.

• R CMD check (local, patched copy): results to follow in a comment.

What I could not verify

• The reported instability itself. On the fixture, the unsorted candidate query returned the same (unsorted) order in 30 of 30 runs with 4 threads. I could not reproduce two identical calls disagreeing. The fix holds by construction, and the tests pin the sorted order, but I have not seen the original symptom.
• The issue's timings (about 0.15 s to build suggestions vs about 0.55 s to repeat both verbs). These need the live corpus, which I had no access to.
• Nothing ran against the live corpus.

Noticed, not changed

• .attach_ig_counterparts() takes the first exact-set match from an unordered harmonization_recipes query. On the fixture no direct-expenditure recipe has more than one match, so ig_recipe_id is deterministic there. If the live catalog ever gained two matches, the choice would be arbitrary.
• The package does not enable roxygen Markdown (Roxygen: list(markdown = TRUE) is absent), so backticks and [links] render literally in every .Rd page, including this one.
• roxygen warns @noRd must not be followed by any text at R/manifest.R:130 and R/views.R:124.

---

Generated by Claude Code

[github-actions]

Thanks for this — and one thing worth knowing before it happens.

This repository is a mirror. Development happens on Gitea at
gitea.civilytics.org/Civilytics/uscogdata. Your pull request will be fetched
from here, landed there, and synced back.

Because that merge preserves your commits at their original SHAs, GitHub will
mark this pull request "Merged" on its own — without anyone visibly clicking
the Merge button, and possibly without a review comment on this page first.

If your pull request closes as "Merged" and nobody appears to have merged it,
that is the normal, successful outcome — not a rejection.

If it is not going to be merged, you will get an actual reply saying so.

Substantial contributions get a ctb entry in DESCRIPTION, which surfaces in
citation("uscogdata"). There is no CLA and no DCO sign-off.

Full details: CONTRIBUTING.md.