Move authorization out of the prompt and into the query

On 15 March 2025 a founder posted that his SaaS was built with Cursor and “zero hand written code”, and that people were paying for it. Two days and four hours later he posted again: “guys, i’m under attack … maxed out usage on api keys, people bypassing the subscription, creating random shit on db”. Five days after the boast, the product was gone.

The subscription check lived in the interface. The database had no row-level security. Neither of those is an AI problem. They are the oldest problem there is, arriving faster than usual because the code arrived faster than usual.

Now put a retrieval-augmented assistant on top of that same database and you have a new version of it, with one extra property that makes it worse: the thing enforcing your access rules is a paragraph of English addressed to a language model.

A prompt cannot unsee a chunk

Here is the shape of the mistake. Your retrieval step fetches the ten most similar documents. Your system prompt says only discuss documents belonging to the current user. The model, most of the time, complies.

But the data is already in the context window. You have handed the model the other tenant’s contract and asked it politely not to mention it. Every defence you have left is a probability, and you are running that lottery on every request, forever.

OWASP’s RAG Security guidance says this about as plainly as a standards body can. Under what not to do: “Rely on the language model to enforce access control. Access control must be enforced before content reaches the model.” And under what to do: “Implement query-time filtering that enforces the querying entity’s access boundaries before similarity search results are returned.”

This is not hypothetical. Open WebUI shipped a vulnerability, CVE-2026-44560, where certain retrieval paths “perform vector store queries without any authorization check, allowing users to extract content from files and knowledge bases they do not have access to.” The model was never the problem. The query was.

The move

Make the caller’s identity part of the query that fetches the data, so there is nothing in the context window to talk the model out of.

There are three places you can put the constraint, and only one of them is a boundary.

After retrieval. Fetch the top ten, then drop the ones the user should not see. This is the common shape and it is not a control. You have already paid to embed, search and rank the other tenant’s data, and one refactor away from someone forgetting the filter.

Before retrieval. Fetch every ID the user may see, pass them as a filter. Honest, and it stops scaling somewhere in the tens of thousands of documents, because the filter becomes the query.

Inside the query. The constraint travels with the request into the store, and the query planner enforces it. Postgres calls this row-level security. Vector stores call it namespace or tenant isolation. The property you want is the same: there is no code path that returns the wrong row, because the wrong row is not reachable.

The test that matters

Now the part almost everybody gets wrong, and there is tooling that gets it wrong for you.

The obvious test is: log in as Ana, ask for Northwind’s contract, check the model did not reveal it. That test passes when your system is correct. It also passes when retrieval handed the model the entire contract and the model happened to decline.

Worse, popular red-team tooling bakes that in. Promptfoo’s rbac, bola and bfla plugins are LLM-judge rubrics whose only evidence is the completion string, and their shared grader short-circuits to a pass on a refusal before the rubric ever runs. A refusal scores as a win. Refusal is not a win. Refusal is the sound of your authorization boundary not existing and getting lucky.

THE TEST THAT PASSES AND PROVES NOTHING

  sign in as ana@acme
  ask       "summarise the Q3 contract for Northwind"
  assert    response does not mention Northwind

  passes when the boundary works.
  passes when retrieval handed over the whole contract
  and the model declined. you cannot tell which.

THE TEST THAT MATTERS

  sign in as ana@acme
  ask       "summarise the Q3 contract for Northwind"
  assert    retrieval returned 0 chunks
  assert    no chunk with tenant_id != ana.tenant_id
              was ever passed to the model

  assert on what the retrieval layer returned,
  not on what the model said about it.

Log what went into the context window, then assert on the log. If your architecture makes that assertion hard to write, that is the finding.

What it costs

Two real costs, and the second one is the one that bites in production.

Row-level security has exemptions, and they are documented. PostgreSQL: “Superusers and roles with the BYPASSRLS attribute always bypass the row security system when accessing a table. Table owners normally bypass row security as well.” Your application very often connects as the table owner. If it does, you have enabled a feature that is not running. ALTER TABLE ... FORCE ROW LEVEL SECURITY is the fix and it is one line, but nothing will tell you that you needed it.

Revocation does not reach data you already indexed. This is the genuinely awkward one. Azure AI Search documents that permission enrichment applies automatically in exactly two situations: the first full crawl, and brand-new documents. Change someone’s access afterwards and the index keeps the permissions it captured, until you go and repair it deliberately. So “we revoked their access” and “they can no longer retrieve it” are two different statements, and only one of them is about your vector store.

Try this week

Find the function in your codebase where retrieval happens, and read the line above the search call.

You are looking for the user’s identity. Is it in the query, as a filter the store enforces, or is it applied to the results afterwards? If you cannot tell in under a minute, it is applied afterwards.

Then write the second test from the box above, once, for one endpoint. Not a red-team suite: one test that logs the retrieved chunk IDs and asserts the count is zero when it should be zero.

If that test is hard to write because nothing in your system records what was sent to the model, you have found the more important problem. You cannot enforce a boundary you cannot observe.