Address review

[magik6k]

No description provided.

[Copilot]

Pull request overview

This PR updates the Filecoin Gateway/RIBS stack to improve staging (LocalWeb) behavior, diagnostics surfaced to the web UI, and robustness around wallets, provider selection, and DB startup—along with a set of tests and documentation updates.

Changes:

• Tighten multipart/S3 validation and listing behavior (including temporary object listing and multipart upload state checks).
• Add operational diagnostics (deal loop stats, provider cooldowns, CAR upload stats) and expose them via RPC + web UI.
• Improve LocalWeb staging configuration/docs, container/test harness setup, and DB startup retry logic.

Reviewed changes

Copilot reviewed 60 out of 67 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
test/testcontainers.go	Creates/mounts a temp wallet for containerized integration tests and cleans it up on teardown.
server/s3frontend/server.go	Introduces a multipartUploadLookup interface and rejects inactive/expired multipart uploads.
server/s3frontend/server_test.go	Adds tests ensuring inactive/expired multipart uploads return 404.
server/s3/handlers.go	Adds range validation for multipart query params (partNumber, max-parts, part-number-marker).
server/s3/handlers_test.go	Adds tests for newly enforced multipart parameter ranges.
README.md	Updates LocalWeb staging guidance, modes, and docker run instructions/paths.
rbstor/index_cql.go	Adds atomic estimated size tracking for CQL index and initializes estimate from DB count.
rbstor/index_cql_test.go	Extends EstimateSize test to verify decrement on DropGroup.
rbstor/diag.go	Preallocates writable group slice capacity for diagnostics output.
rbdeal/wallet.go	Treats certain chain “not yet visible” errors as transient and avoids failing WalletInfo in those cases.
rbdeal/ribs.go	Refactors wallet default resolution; adds OpenExistingWallet helper; initializes new diagnostic/cooldown state.
rbdeal/ribs_wallet_test.go	Adds tests for OpenExistingWallet / OpenOrCreateWallet behavior.
rbdeal/provider_cooldown.go	Adds provider cooldown/backoff logic based on rejection/connectivity errors and exposes it in provider diagnostics.
rbdeal/provider_cooldown_test.go	Adds tests for cooldown attempt bumping, expiry/clearing, and diagnostic application.
rbdeal/group_deal.go	Adds LocalWeb pre-deal transfer probe; filters providers on cooldown; records/clears cooldown on failures/success.
rbdeal/external.go	Extends ExternalOffloader with PreDealTransferCheck.
rbdeal/external_s3.go	Implements PreDealTransferCheck (no-op) for S3 offload.
rbdeal/external_metered.go	Pass-through implementation for PreDealTransferCheck on metered wrapper.
rbdeal/external_http_path.go	Implements PreDealTransferCheck via HTTP HEAD probe with status/size checks.
rbdeal/external_http_path_test.go	Adds tests validating LocalWeb pre-deal transfer probe behavior.
rbdeal/deal_tracker.go	Adds loop backoff on failures and records deal loop stats for diagnostics.
rbdeal/deal_diag.go	Exposes reachable-provider cooldown diagnostics, deal loop stats, and CAR upload stats.
rbdeal/deal_db.go	Expands provider metadata query + recent deal scanning; adds GetGroupByExternalPath.
rbdeal/car_server.go	Removes JWT auth, serves CARs by randomized path lookup, and adds request/byte counters.
integrations/web/rpc.go	Exposes CAR upload stats and new deal loop stats via RPC.
integrations/web/ribswebapp/src/routes/WritableGroups.js	Hardens against non-array responses; uses derived groups for rendering.
integrations/web/ribswebapp/src/routes/Status.js	Adds deal loop stats + richer CAR upload stats display, charts, and UI improvements.
integrations/web/ribswebapp/src/routes/Status.css	Adds segmented-control styling and chart/tile layout styles.
integrations/web/ribswebapp/src/routes/Root.js	Updates header branding text.
integrations/web/ribswebapp/src/routes/Root.css	Adds new theme variables and updates branding colors/styles.
integrations/web/ribswebapp/src/routes/Providers.js	Adds feature labels, cooldown column, and improved derived stats.
integrations/web/ribswebapp/src/routes/Providers.css	Adds muted cooldown reason styling.
integrations/web/ribswebapp/src/routes/Provider.js	Enhances provider details with cooldown, retrieval stats, transfer progress, and links.
integrations/web/ribswebapp/src/routes/Provider.css	Adds progress bar + badges + subtle/warning text styles.
integrations/web/ribswebapp/src/routes/Group.js	Fixes external link URL and adds rel="noreferrer".
integrations/web/ribswebapp/src/components/RequestThroughputChart.js	Disables chart animation for smoother high-frequency updates.
integrations/web/ribswebapp/src/components/LatencyDistributionChart.js	Disables chart animation for smoother high-frequency updates.
integrations/web/ribswebapp/src/components/IOThroughputChart.js	Disables chart animation for smoother high-frequency updates.
integrations/web/ribswebapp/src/components/ErrorRateChart.js	Disables bar animation for smoother updates.
integrations/web/ribswebapp/public/index.html	Updates theme color + page title branding.
integrations/web/ribswebapp/build/static/js/main.56524901.js.LICENSE.txt	Updates build artifact license bundle (new build output).
integrations/web/ribswebapp/build/static/css/main.78016276.css.map	Updates build artifact sourcemap (new build output).
integrations/web/ribswebapp/build/static/css/main.78016276.css	Updates build artifact CSS (new build output).
integrations/web/ribswebapp/build/static/css/main.23f311b7.css.map	Removes old build artifact sourcemap.
integrations/web/ribswebapp/build/static/css/main.23f311b7.css	Removes old build artifact CSS.
integrations/web/ribswebapp/build/index.html	Updates build artifact HTML to point at new bundles + branding.
integrations/web/ribswebapp/build/asset-manifest.json	Updates build artifact manifest for new hashed asset names.
integrations/kuri/ribsplugin/s3/object_index_cql.go	Adds temporary-only listing and persists/exposes expires_at in object scans.
integrations/kuri/ribsplugin/s3/bucket.go	Uses ListTemporary for multipart part object listing.
integrations/gwcfg/wallet.go	Consolidates transient chain-visibility checks; makes FIL faucet request amount explicit.
integrations/gwcfg/setup.go	Improves onboarding flow: staging mode selection, endpoint tests, CIDGravity policy verification, and post-setup menu.
integrations/gwcfg/opts.go	Adds CIDGravity service URL option for onboarding-policy setup.
integrations/gwcfg/main.go	Adds logger quieting and homedir expansion helper usage in setup tooling.
integrations/gwcfg/cidgravity.go	Adds CIDGravity service endpoints to initialize/test onboarding policy; improves response body closing.
iface/s3.go	Extends S3ObjectIndex with ListTemporary.
iface/iface_ribs.go	Exposes CAR upload stats + deal loop stats diagnostics and extends provider/deal metadata.
docker-compose.yml	Adjusts ports, adds restart policies, mounts car staging dir, and makes kuri init conditional.
database/sqldb/db_yugabyte.go	Adds retry/backoff for Yugabyte SQL ping + migrations during startup.
database/cqldb/cql_db_yugabyte.go	Adds retry/backoff for YCQL migrations/session creation and closes migration session.
configuration/config.go	Adds LocalWeb validation + defaults and validates replica counts are >= 1.
configuration/config_test.go	Updates LocalWeb URL expectations, adds LocalWeb mode validation tests, and replica count validation tests.
carlog/idx_wal.go	Switches to RWMutex and adds read locks to make WAL index concurrent-reader safe.
carlog/idx_wal_test.go	Adds a concurrency test for readers during Put.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.