Repository navigation
π± Top up v6 with v5 (ETag cache size setting and 7 more) - #17290
Conversation
The Copilot usage sniff (crux/proxy/github_proxy_copilot.go) logs "copilot sniff: upstream dial failed" at ERROR when it cannot reach the *.githubcopilot.com host it is MITMing - e.g. a dial timeout to telemetry.enterprise.githubcopilot.com. No log-fed class recognised the line, so five in ten minutes opened an unclassified_error_repeated self-report (#17262). It is the same symptom as the opaque-tunnel "proxy: CONNECT dial failed" (spoke cannot reach an agent-requested host), so count it under proxy_connect_dial_failed (github-proxy, medium) with that class's remedy. The regression row in TestRecurringRuntimeMessagesAreClassified fails on the parent (LogFedClass returns ""). Fixes #17262 Hive-Run: #17262 Hive-Plan: selfreport-unclassified-error-classification Hive-Spec: fleet-report#log-fed-classes Signed-off-by: Karibhou <Karibhou@users.noreply.github.com> Co-authored-by: Karibhou <Karibhou@users.noreply.github.com> Co-authored-by: hivecommons-hive[bot] <280983584+hivecommons-hive[bot]@users.noreply.github.com>
The one document a new driver wires up from (RFC crux#114 order-of-work item 7): S1βS7 composed end to end with curl and crux CLI forms, server config keys and flags, control-token vs lane-principal auth, the failure drills from the acceptance sections, a symptom β saved-record map, and what hive uses today vs. after the 2026-11-16 cutover. Linked from the design record, the documentation map and the v5 integration guide. Part of hivecommons/crux#114 Signed-off-by: Andrew Anderson <andy@clubanderson.com>
Automated release commit. Compiles changelog.d/ fragments and moves the CHANGELOG.md Unreleased section into a dated v5.204.0 entry. See src/docs/releases.md. Signed-off-by: hive-release-bot <actions@github.com>
release: v5.204.0
* β¨ crux: saved run results (S7 complete)
Completes RFC crux#114 S7 beyond the minimum that shipped with S2:
- POST /v1/runs/{id}/restart re-runs the saved request as the original
owner, re-validated against the current configuration; the two
records link both ways; 409 conflict while the old run is live or
when its record predates the saved request; run.restart audit.
- GET /v1/runs lists runs newest first, owner-scoped unless operator,
with ?status= and ?limit=; output is stripped from the list.
- GET /v1/runs/{id}/history is the record's status transitions. The
runstate index now logs transitions only (payload-only updates are
not transitions).
- Reconcile at serve restart marks running steps of an unfinished run
unfinished with the same reason.
- crux runs list|restart|history.
Part of hivecommons/crux#114
Signed-off-by: Andrew Anderson <andy@clubanderson.com>
* π crux serve: S7 routes in the route table
Signed-off-by: Andrew Anderson <andy@clubanderson.com>
* π crux design: drop forward link to the integration guide
Signed-off-by: Andrew Anderson <andy@clubanderson.com>
* crux/api: cover the saved-run client and list query
Signed-off-by: Andrew Anderson <andy@clubanderson.com>
* π crux design: link the integration guide
Signed-off-by: Andrew Anderson <andy@clubanderson.com>
* crux/api: run paths do not escape ids
Signed-off-by: Andrew Anderson <andy@clubanderson.com>
---------
Signed-off-by: Andrew Anderson <andy@clubanderson.com>
Automated release commit. Compiles changelog.d/ fragments and moves the CHANGELOG.md Unreleased section into a dated v5.204.1 entry. See src/docs/releases.md. Signed-off-by: hive-release-bot <actions@github.com>
release: v5.204.1
β¦ default to 128 MB (#17286) * β¨ Make the GitHub ETag response cache size configurable and raise the default to 128 MB The conditional-request (ETag / 304) cache was hard-coded at 4096 entries / 64 MiB. A busy multi-repo hive filled it within hours; from then on every store evicted a body another poller was about to revalidate, so free 304s became charged fetches and hourly REST usage crept back toward the installation limit. The operator reached for HIVE_GITHUB_ETAG_CACHE_MAX_MB, which did not exist. - github.etag_cache_max_mb / github.etag_cache_max_entries (YAML, JSON, env HIVE_GITHUB_ETAG_CACHE_MAX_MB / _MAX_ENTRIES), validated 0..2048 MiB and 0..1Mi entries with 0 meaning the default. - Defaults raised to 128 MiB / 8192 entries; bodies above 512 B are stored gzip-compressed so the byte cap holds several times as many responses. - github.SetETagCacheLimits resizes the shared cache live (evicting LRU at once when shrinking); called at boot and from PUT /api/config/github. - ETagCacheInfo / /api/gh-rate-limits / status etag_cache report bytes, max_bytes and max_entries; the Debug GitHub API budget panel shows an 'ETag cache: N entries Β· X / Y MB' row. - Settings β Forge App gains 'ETag cache size (MB)' and 'ETag cache entries' with help marks; docs, hive.yaml.example and env-vars reference updated. Signed-off-by: Andrew Anderson <andy@clubanderson.com> * π± Use the shared formatNumber helper for the ETag cache entry count Signed-off-by: Andrew Anderson <andy@clubanderson.com> --------- Signed-off-by: Andrew Anderson <andy@clubanderson.com>
Brings v5.204.0/v5.204.1 and the configurable GitHub ETag cache size (#17286) onto v6. ETag cache: v5's config-driven bounds and gzip store are taken for the shared code, with v6's eviction counter, ETagCacheSnapshot, cacheableSince and App-JWT transport chain kept on top. Signed-off-by: Andrew Anderson <andy@clubanderson.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
There was a problem hiding this comment.
Read the merged src/pkg/github/etagcache.go at 985424a end to end plus its callers (cmd/hive/github_client_config.go:126, pkg/dashboard/api_config.go:357, pkg/config/load.go:450-459, forge_config.go:216-241). The cache conflict resolution looks correct to me: byte accounting uses len(e.body) consistently in get/put/drop/evictLocked (etagcache.go:340-399), the v6-only evictions counter is incremented at :396 and surfaced through ETagCacheSnapshot (:184-198), and the dropped in-package env read is genuinely replaced β load.go:450 reads HIVE_GITHUB_ETAG_CACHE_MAX_MB into GitHub.ETagCacheMaxMB, validate.go:162 bounds it, and boot applies it via SetETagCacheLimits at github_client_config.go:126.
One merge residue to fix before this lands (docs-currency, low):
src/docs/env-vars.md now has two rows for HIVE_GITHUB_ETAG_CACHE_MAX_MB. Line 60 is v5's (default 128, 1β2048, applies from Settings) and is correct for the merged code. Line 119 is v6's pre-#17286 row and is now wrong on three counts: it says the default is 64 (code: forge_config.go:208 = 128), that non-positive values "fall back to the default" (code: validate.go:162 rejects < 0 at boot), and "read once at process start" (code: api_config.go:357-358 re-applies on a Settings save). Neither branch had both rows (v5: 1, v6: 1, this head: 2), so it's an auto-merge artifact; env_vars_doc_parity_test.go only checks presence, which is why CI is green. Deleting line 119 is the whole fix.
Info, not a defect: v6's etagCacheMaxBytesFromEnv silently accepted any value up to 1<<20 MiB; the merged code refuses > 2048 at Validate (forge_config.go:239). That is v5's documented choice (env-vars.md:60), just noting it in case a v6 operator has an oversized value set β boot will now fail loudly instead of honouring it.
Checked nothing else in the 54 files beyond the conflict-table entries; DCO/native-queue-policy reds match the stated forward-merge pattern.
β hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
Hive status β updated in place; edits do not notify. Full history is in the Activity Monitor. Review findings from Read the merged tree at 1. Hub mode loses 2. Everything else I checked holds: @clubanderson this PR is held at Why:
Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path. |
|
Diagnosis (hive scanner, read-only β maintainer branch, nothing pushed): The only real red is Why probot DCO is red: it flags carried Why the policy did not substitute prow's Options, all maintainer-side:
Tide is otherwise waiting only on π Hive Agent: β hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 |
|
Thank you for your contribution! Your PR has been merged. We'd love to hear how your experience was: share feedback |
Fifth top-up after #17278: forward-merge of
v5(d8b1d3a78) intov6. Plain merge commit β merge with--merge, not squash β so v6 keeps v5's SHAs and sign-offs. Supersedes the automatedsync/v5-to-v6PR #17282.Commits carried (8)
d8b1d3a78β¨ Make the GitHub ETag response cache size configurable and raise the default to 128 MB (β¨ Make the GitHub ETag response cache size configurable and raise the default to 128 MBΒ #17286)fc5612576Merge pull request release: v5.204.1Β #17289 from hivecommons/release-gate/v5.204.101b73d6b2π release: v5.204.1573eb034bβ¨ crux: saved run results (S7 complete) (β¨ crux: saved run results (S7 complete)Β #17281)fbd8e1905Merge pull request release: v5.204.0Β #17285 from hivecommons/release-gate/v5.204.06880c18a8π release: v5.204.0155d51f12π crux: integration guide for work services (π crux: integration guide for work servicesΒ #17283)c05ba93acπ fleetreport: classify copilot sniff upstream dial failures (π fleetreport: classify copilot sniff upstream dial failuresΒ #17263)changelog.d/added-etag-cache-size-setting.mdandadded-crux-s7-saved-run-results.mdare kept for the next v6 release (not compiled on v5 yet).Conflict resolutions
src/pkg/github/etagcache.goSetETagCacheLimits,ETagCacheUsage,gzipped/sizeentry fields, body-only byte accounting); v6-only pieces kept on top:evictionscounter +ETagCacheSnapshot(now also reportingMaxEntries),cacheableSince,githubAppJWTTransportChain. v6's in-packageHIVE_GITHUB_ETAG_CACHE_MAX_MBread dropped β the same env var is now handled byconfig.applyBootstrapEnvand applied viaSetETagCacheLimitsat boot.TestETagCache_MaxBytesFromEnvremoved accordingly;etagcache_workingset_test.goupdated to the v5 field names.src/pkg/github/client.goETagCacheInfois the union (bytes,max_bytes,max_entries,evictions);etagCacheInfo()delegates toETagCacheSnapshot().src/pkg/dashboard/status_builder.go,src/pkg/hub/github_http.gosrc/pkg/config/config.goGitHubConfigfields +DefaultGitHubETagCacheMax*consts +Effective*/ValidateETagCacheLimitsβforge_config.go, env override βload.go;validate.goauto-merged.CHANGELOG.mdsrc/docs/scale-envelope.mdclient.go:5489-5499;check-docs-citations.pyclean (2031 verified).src/pkg/dashboard/static/index.htmlregenerated viago generate ./pkg/dashboard/...(-checkpasses).src/scriptsunchanged, noTRUSTED_SCRIPTS_TREEre-pin needed.go build ./...passes.DCO/
native-queue-policyreds are expected on forward-merges (others' v5 commits); the merge commit itself is signed off. To be admin-merged with a merge commit like #17265.