Repository navigation
feat: add BudgetGuard middleware for cost control - #36
Open
oziriemeka wants to merge 1 commit into
Open
oziriemeka wants to merge 1 commit into
oziriemeka wants to merge 1 commit into
Conversation
Member
|
Hi @oziriemeka, can't thank you enough for this contribution, I'm taking some time to go through the PR. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Production AI applications need reliable safeguards against unexpected costs caused by excessive token consumption, model changes, or misuse.
Previously, the Intercept middleware stack did not provide a mechanism for enforcing spending limits at the request, session, or user level.
This PR introduces
promptphp/intercept-budget-guard, a middleware package that estimates generation costs before invoking the AI provider and enforces configurable spending budgets.Key Features
Pre-request cost estimation: Estimates the cost of a generation step before making an AI provider call.
Configurable spending limits: Supports per-request, per-session, and per-user-per-day budgets.
Flexible enforcement actions:
block— Rejects requests that exceed the configured budget.log— Logs budget violations while allowing requests to proceed.truncate— Reduces the latest user message to fit within the available budget, falling back toblockwhen necessary.Unknown model handling: Configurable behaviour when pricing information is unavailable.
Custom callbacks: Allows consumers to override the default enforcement action.
Persistent budget tracking: Tracks accumulated spending across requests using Laravel's cache.
Implementation Decisions
The original proposal in #12 referred to the middleware as
CostGuardand suggested usingRateLimitExceededException.The implementation uses
BudgetGuardandBudgetGuardExceptioninstead, aligning with the package name reserved inROADMAP.mdand the existing<Package>Exceptionnaming convention across the middleware collection.Additionally, rather than using Laravel's
RateLimiter, spending is tracked directly through the cache.Laravel's
RateLimiteris designed around counting attempts within a time window, whereas budget enforcement requires tracking monetary values.To support this, costs are represented as **integer micro-dollars**, avoiding floating-point inaccuracies and enabling atomic increments on supported cache drivers. Budget windows follow the same timer-key approach used internally by Laravel's `RateLimiter`.
Closes #12.
Dependencies
No new root-level Composer dependencies are introduced.
The new package depends on:
promptphp/intercept-support— existing sibling package within the monorepo.laravel/ai: ^1.0— consistent with the other middleware packages.Install dependencies locally using:
Testing
Automated Tests
The implementation adds 22 tests, bringing the full suite to 264 passing tests.
Command | Purpose | Result -- | -- | -- composer test | Full test suite | 264 passing vendor/bin/pest src/BudgetGuard/tests | BudgetGuard tests | 22 passing composer test:lint | Laravel Pint | Passing composer test:types | PHPStan Level 5 | Passing composer packages:validate | Monorepo package validation | PassingTest Coverage
The new tests cover:
Allowing requests that remain within their configured budgets.
Blocking requests whose projected costs exceed the per-request budget, including costs accumulated during earlier generation steps.
Logging budget violations without interrupting execution.
Truncating the latest user message to fit the remaining budget.
Falling back to
blockwhen truncation cannot produce a sufficiently inexpensive request.Handling models with unavailable pricing according to
unknownModelAction.Overriding the default enforcement action through a custom callback.
Tracking per-session and per-user-per-day spending across separate agent runs.
Skipping session or user budget enforcement when the corresponding key resolver does not resolve a value.
Rejecting invalid enforcement actions and non-positive numeric configuration values during construction.
Manual Verification
The middleware can also be verified end-to-end using a Laravel application with
laravel/ai.Create a temporary Laravel application and install
laravel/ai.Configure a Composer path repository pointing to this monorepo's
src/*packages.Install the new package:
Generate an agent using
php artisan make:agent, add theHasMiddlewaretrait, and configure:Invoke the agent:
Expected: A
BudgetGuardExceptionis thrown before any provider API call. No provider API key is required to verify this behaviour.Change the action to
log, retaining the same budget.Expected: The violation is recorded in
storage/logs/laravel.log, and execution proceeds to the provider call.Change the action to
truncateand provide a sufficiently long prompt with a budget that permits a shortened request.Expected: The middleware shortens the prompt and allows execution to continue. If the available budget remains insufficient after truncation, it falls back to
block.Summary
This PR introduces configurable, middleware-level cost controls for AI generation requests, providing applications with a way to manage spending before provider calls are made.
The implementation follows the existing Intercept package conventions, introduces no new root-level dependencies, and includes automated coverage for budget enforcement, accumulated spending, truncation, and configuration validation.