Put a hard per-user cost ceiling in front of the model

You were not careless. You went into the billing settings and set a limit, and you have an alert configured, and you would describe your AI feature as having cost controls.

Now read what you actually bought.

OpenAI’s rate limit documentation is unambiguous: “Rate limits are defined at the organization level and at the project level, not user level.” Spend limits are the same shape, configured for an organization or a project. Anthropic’s tier caps are enforced at the organisation level, with workspace limits available if you set them yourself.

There is no per-user ceiling on sale. Every ceiling your provider offers is a ceiling on everybody, which means it is a ceiling that one person is entitled to consume in full.

Your budget alert is not a control, and its vendor says so

Google Cloud’s budget documentation carries this as a highlighted caution: “Setting an alerts-only budget doesn’t automatically cap Google Cloud or Google Maps Platform usage or spending.” The next sentence removes any remaining ambiguity: budget alert emails “might prompt you to take action to control your costs, but they don’t automatically prevent the use or billing of your services” when the threshold is crossed.

Then there is the timing. From the same page: “After you create a budget, it may take several hours before receiving the first email or Pub/Sub notification.”

Put those together honestly. You have an email, which arrives some hours after a threshold you have already crossed, telling you about money that is already gone, and which stops nothing on its way past. That is a smoke detector wired to a postcard.

The move

Estimate the cost before you dispatch, decrement a counter, and refuse with a 429 that says what is left.

The order is the whole thing. Every control described above acts after the provider has done the work and priced it. Yours has to act before, because after is where the money already is.

How it actually works

Four steps, and none of them is exotic.

Estimate from the request’s shape. You know the model, the prompt length, the history you are about to attach and your configured maximum output. That gives you an upper bound on cost before you call anybody.

Reserve it. Decrement the user’s remaining budget by the estimate, atomically, before dispatch.

Dispatch, or refuse. If the reservation would take them past their ceiling, do not call the provider at all. That is the step that makes this a control rather than a report.

Reconcile afterwards. The response comes back with real usage. Replace your estimate with the actual number and return the difference.

If that sounds like a lot of machinery to build, it is a shipped pattern rather than an invention. LiteLLM’s proxy has done exactly this since May 2026, on by default, and its documentation describes the mechanism plainly: it “estimates the request’s maximum cost from the request body and the model’s pricing”, “temporarily reserves that amount against the applicable budget”, and “If the reservation would exceed the budget, LiteLLM rejects the request before sending it to the provider.” Then: “After the response is priced, LiteLLM replaces the reservation with the actual cost.”

The fourth step is the one people skip, and skipping it is expensive in both directions. An estimate that is never corrected downwards slowly strangles your heaviest legitimate users. One never corrected upwards leaks money quietly for months.

BEFORE                            AFTER

request                           request
  |                                 |
  |                                 estimate max cost from
  |                                   model + prompt + history + max_tokens
  |                                 reserve against this user's budget
  |                                   |- would exceed?  429, no provider call
  |                                   |
  +--> provider                       +--> provider
  |                                 |
  response                          response
  |                                 reconcile: replace estimate with real usage
  |
  billing export, hours later
  budget alert email, later still
  you, next morning

Say what is left

A refusal that only says “no” turns one user’s exhausted budget into a support ticket and a bug report.

RFC 6585 defines 429 as indicating “that the user has sent too many requests in a given amount of time”, and says the response “SHOULD include details explaining the condition, and MAY include a Retry-After header indicating how long to wait before making a new request.” Do both. Send the remaining budget, the reset time, and a sentence a human can read.

The RFC also notes something worth keeping in mind: “this specification does not define how the origin server identifies the user, nor how it counts requests.” That is your decision, and it is the decision that matters. Per authenticated account is the obvious answer. Per IP is not an answer, because the people you are worried about have more than one.

What it costs

Your estimate is always wrong. You are bounding output tokens you have not generated yet, so you will reserve more than you spend, most of the time. Without reconciliation that error accumulates against your most active users, who are usually the ones you least want to annoy.

A ceiling denies service to somebody eventually, and one day it will be a real customer having an unusually productive afternoon. Set the number where a legitimate heavy day passes and an automated loop does not, then instrument how often it fires. If it never fires, it is decorative. If it fires on people you would have wanted to serve, it is too low, and you can only learn that from the log.

And it is one more thing between the user and the answer. Keep the reservation in one fast store, and fail open on infrastructure errors rather than locking everyone out because your counter is unreachable. A budget check that takes your product down has cost you more than the runaway loop would have.

Try this week

Answer one question about your AI feature: what is the most a single user could spend today if they tried?

If the answer is your organisation’s entire limit, you have no per-user ceiling, and almost nobody does by default.

Then ship the cheapest version. Not the estimator, not reconciliation: a counter. Count requests per user per hour in whatever fast store you already run, and refuse over a threshold with a 429 that names the reset time. Requests are a bad proxy for cost and it will take you an hour.

It is also the difference between a bad day that costs you a number you choose, and one that costs you a number somebody else chooses.