Skip to content

feat(tasks): report priced sandbox compute credits - #73031

Closed
adboio wants to merge 7 commits into
posthog-code/trusted-task-billing-provenancefrom
posthog-code/sandbox-compute-credits
Closed

feat(tasks): report priced sandbox compute credits#73031
adboio wants to merge 7 commits into
posthog-code/trusted-task-billing-provenancefrom
posthog-code/sandbox-compute-credits

Conversation

@adboio

@adboio adboio commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Problem

Sandbox compute needs a priced usage counter without allowing a client-selected task origin to determine whether work is charged.

Changes

This PR is stacked on #74491. It calculates integer compute credits from the snapshotted sandbox ledger and applies the existing origin rules. User-created work is billable only when SandboxSession.compute_source is the server-verified posthog_desktop value. Signal-report work remains excluded, and loop billing uses the provision-time internal classification.

Resource quantities use the configured burstable request floors, durations round up to whole seconds, and pricing remains isolated behind provisional named constants.

How did you test this code?

  • Ruff lint and formatting for the changed aggregation and tests
  • Regression coverage for verified Desktop source, default-denied sources and origins, signal reports, direct app origins, loop classification, TTL and reporting-window boundaries, and burstable pricing

Database-backed tests rely on CI because the configured Postgres hostname is unavailable locally.

Automatic notifications

  • Publish to changelog?
  • Alert Sales and Marketing teams?

Docs update

No user-facing documentation change.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Codex implemented the counter with direct human design review. Repository testing and Django migration guidance shaped the validation. The key decision is to bill from immutable ledger provenance rather than the mutable task or a client assertion.


Created with PostHog Code

@adboio adboio self-assigned this Jul 22, 2026
@trunk-io

trunk-io Bot commented Jul 22, 2026

Copy link
Copy Markdown

Merging to master in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@adboio adboio added the skip-inkeep-docs Use this label to skip an Inkeep docs PR in posthog.com label Jul 22, 2026 — with PostHog
@github-actions

Copy link
Copy Markdown
Contributor

Hey @adboio! 👋

It looks like your git author email on this PR isn't your @posthog.com address (adambowker98@gmail.com). Since you're on the PostHog team, it's worth pointing your local git author email at your @posthog.com address. Why it matters:

  • Consistent work identity in git history — internal tooling that attributes commits to team members keys off your @posthog.com address.
  • Keeps team contributions easy to tell apart from external community ones when scanning history.

You can fix it for this repo with:

git config user.email "you@posthog.com"

Or set it globally with git config --global user.email "you@posthog.com". No need to redo this PR — just a nudge for next time. 🙂

@posthog

posthog Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

🦔 ReviewHog reviewed this pull request

Found 0 must fix, 1 should fix, 0 consider.

Published 1 finding (view the review).

@posthog

posthog Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

ReviewHog Alpha 🦔 If you find any issues helpful - please reply "valid", "invalid", etc., for evaluation purposes 🙏

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ReviewHog Report

Changes

Issues: 1 issue

Files (4)
  • posthog/tasks/usage_report.py
  • posthog/temporal/usage_report/queries.py
  • products/tasks/backend/logic/services/sandbox_usage.py
  • products/tasks/backend/models.py

Comment thread products/tasks/backend/logic/services/sandbox_usage.py
@adboio
adboio requested review from a team and pawel-cebula July 23, 2026 00:16
@adboio
adboio marked this pull request as ready for review July 23, 2026 00:16
@pr-assigner-resolver-posthog
pr-assigner-resolver-posthog Bot requested a review from a team July 23, 2026 00:16
@greptile-apps

greptile-apps Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor
Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
products/tasks/backend/logic/services/sandbox_usage.py:156
**Historical Loop Usage Becomes Unbillable**

When a non-internal loop is later marked internal or deleted, this annotation reads `True` or `NULL` instead of the loop state when the sandbox ran. Rebuilding an earlier report then drops credits for compute that was billable at execution time, so historical billing changes after the session is complete.

### Issue 2 of 2
products/tasks/backend/logic/services/sandbox_usage.py:156
**Internal Loop Usage Becomes Billable**

When an internal loop is later marked non-internal, this annotation reads the new `False` value and the billing gate charges its earlier sandbox sessions. Historical reports can therefore add credits for compute that was internal when it ran; the billing classification needs to be recorded with the session rather than resolved from mutable current state.

Reviews (1): Last reviewed commit: "fix(tasks): expand billable sandbox orig..." | Re-trigger Greptile

# Unscoped: the usage report aggregates across every team in the region.
sessions = (
SandboxSession.objects.unscoped()
.annotate(task_loop_internal=F("task_run__task__loop__internal"))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Historical Loop Usage Becomes Unbillable

When a non-internal loop is later marked internal or deleted, this annotation reads True or NULL instead of the loop state when the sandbox ran. Rebuilding an earlier report then drops credits for compute that was billable at execution time, so historical billing changes after the session is complete.

Prompt To Fix With AI
This is a comment left during a code review.
Path: products/tasks/backend/logic/services/sandbox_usage.py
Line: 156

Comment:
**Historical Loop Usage Becomes Unbillable**

When a non-internal loop is later marked internal or deleted, this annotation reads `True` or `NULL` instead of the loop state when the sandbox ran. Rebuilding an earlier report then drops credits for compute that was billable at execution time, so historical billing changes after the session is complete.

How can I resolve this? If you propose a fix, please make it concise.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed by snapshotting loop_internal onto SandboxSession at provision time and billing from that immutable value. The regression test flips both loop directions after provisioning.

Comment thread products/tasks/backend/logic/services/sandbox_usage.py Outdated
Comment thread products/tasks/backend/logic/services/sandbox_usage.py Outdated
if session.burstable:
assert session.cpu_request_cores is not None
assert session.memory_request_mb is not None
cpu_cores = session.cpu_request_cores if session.burstable else session.cpu_cores

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium: Burst usage is priced at reservation floors

A user can run CPU- or memory-intensive code in a default burstable sandbox while this calculation charges only the 0.5-core/1-GiB reservation floor. The sandbox can consume up to its configured limits and the provider bills the greater of reserved and actual usage, so credits can substantially underreport attacker-driven compute consumption. Use provider-reported actual usage, or conservatively price the configured limits when actual usage is unavailable.

@veria-ai

veria-ai Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

PR overview

This pull request adds reporting of priced compute-credit usage for task sandboxes, including CPU and memory consumption calculations.

One issue has been addressed, but burstable sandbox usage can still be priced at reservation floors rather than actual consumption or configured limits. A user can deliberately drive CPU or memory usage above those floors, causing compute-credit reporting to substantially undercharge their activity while provider costs continue to accrue.

Open issues (1)

Fixed/addressed: 1 · PR risk: 6/10

@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

🤖 CI report

Playwright — all passed

All tests passed.

View test results →

Django migration risk — migration analysis complete

We've analyzed your migrations for potential risks.

Summary: 1 Safe | 0 Needs Review | 0 Blocked

✅ Safe

Brief or no lock, backwards compatible

tasks.0076_add_compute_source
  └─ #1 ✅ AddField
     Adding nullable field requires brief lock
     model: taskrun, field: compute_source
  └─ #2 ✅ AddField
     Adding nullable field requires brief lock
     model: sandboxsession, field: compute_source
  └─ #3 ✅ AddField
     Adding nullable field requires brief lock
     model: sandboxsession, field: loop_internal

📚 How to Deploy These Changes Safely

AddField:

This operation acquires a brief lock but doesn't rewrite the table.

Deployment uses lock timeouts with automatic retries, so lock contention will cause retries rather than connection pile-up.

Last updated: 2026-07-29 16:37 UTC (387599a)

@adboio
adboio force-pushed the posthog-code/sandbox-compute-credits branch from 380ca41 to f5d7435 Compare July 29, 2026 02:27
@github-actions
github-actions Bot requested a deployment to preview-pr-73031 July 29, 2026 02:27 In progress
@github-actions

github-actions Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

🦔 Hogbox preview · ❌ build failed

The preview didn't come up for commit 9aa1279. See the build log for the failing step. It'll retry on the next push.

Previews are optional and never block merging. A failure here is often a hogland or tailnet hiccup rather than anything in your PR, so the check stays green and this comment is the status.

Comment thread products/tasks/backend/logic/services/sandbox_usage.py Outdated
@adboio
adboio changed the base branch from master to posthog-code/trusted-task-billing-provenance July 29, 2026 02:29
@adboio
adboio marked this pull request as draft July 29, 2026 02:29
@adboio
adboio force-pushed the posthog-code/trusted-task-billing-provenance branch from ca874ed to 7484578 Compare July 29, 2026 16:21
adboio added 5 commits July 29, 2026 12:26
Generated-By: PostHog Code
Task-Id: 5737ef14-4436-46b0-8568-8ae9143ca6c7
Generated-By: PostHog Code
Task-Id: 5737ef14-4436-46b0-8568-8ae9143ca6c7
Bill user-created sandbox sessions only when their run was initiated through a verified PostHog Code OAuth application.

Generated-By: PostHog Code
Task-Id: e55f8ea4-cc41-4e76-ad48-896b396e95ce
Use the sandbox ledger's loop classification snapshot so later loop edits cannot change historical compute credits.

Generated-By: PostHog Code
Task-Id: e55f8ea4-cc41-4e76-ad48-896b396e95ce
Generated-By: PostHog Code
Task-Id: e55f8ea4-cc41-4e76-ad48-896b396e95ce
@adboio
adboio force-pushed the posthog-code/sandbox-compute-credits branch from 4f4398d to e984894 Compare July 29, 2026 16:27
adboio added 2 commits July 29, 2026 12:27
Generated-By: PostHog Code
Task-Id: e55f8ea4-cc41-4e76-ad48-896b396e95ce
…hog-code/sandbox-compute-refresh

Generated-By: PostHog Code
Task-Id: e55f8ea4-cc41-4e76-ad48-896b396e95ce
@adboio adboio closed this Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-inkeep-docs Use this label to skip an Inkeep docs PR in posthog.com

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant