Azure OpenAI Cost Control: Provisioned Throughput vs Pay-As-You-Go

Most cloud costs scale with data or infrastructure. An Azure OpenAI bill scales with usage of the model — and usage grows exactly when the project is working. Every new user, every longer conversation, every larger document fed into a prompt, every retry on a failed call adds tokens, and tokens are what you pay for. A successful pilot that rolls out to the whole team can multiply its own bill without anyone changing a line of code.

That is not a reason to avoid Azure OpenAI; it is a reason to understand the cost model before the rollout, not after the invoice. The two ways to pay — pay-as-you-go per token, and provisioned throughput for reserved capacity — behave very differently as volume grows, and the right choice at pilot is usually the wrong choice at scale.

Exact prices change and vary by model and region, so this piece deals in the mechanics and the decision rather than quoting rates you should confirm against your own Azure agreement. The mechanics are what let you predict the bill; the rates only fill in the final number.

Pay-as-you-go (also called standard or consumption) bills per token processed. Input tokens — the prompt, the system message, the retrieved context you send — are priced separately from output tokens, which the model generates, and output is usually the dearer of the two. You pay only for what you actually use, with no reserved capacity and no commitment.

This is the right model for pilots and for spiky, low-volume or unpredictable workloads. If usage is intermittent, paying per call means you pay nothing when nothing is happening, which reserved capacity cannot match. It is also how you learn your real token profile — how many tokens a typical interaction actually costs — which is the information you need to make the PTU decision later.

The weaknesses of pay-as-you-go show up at scale. Cost rises linearly with volume with no volume break, so a high, steady workload pays full rate on every token. And because it runs on shared capacity, latency can vary and very high throughput can hit rate limits. For a heavy production workload, both the cost and the variability become arguments for the other model.

Provisioned Throughput Units (PTUs) reserve dedicated model capacity for a fixed cost per period. Instead of paying per token, you buy a guaranteed throughput — a capacity to process so many tokens per minute — and pay the same whether you use all of it or none. It is the reserved-capacity model applied to AI inference.

The two things PTUs buy are predictability of cost and predictability of performance. The bill is a known periodic number regardless of how busy the workload is, which finance teams value far more than a variable line. And because the capacity is dedicated, latency is consistent and you are not competing on shared infrastructure — which matters for a customer-facing or latency-sensitive application. There is usually a minimum commitment, so PTUs are a commitment, not a dial you nudge.

The risk with PTUs is the mirror of pay-as-you-go: you pay for the reserved capacity whether or not you use it, so under-utilised PTUs are pure waste. Buy capacity for a workload that is not yet steady, or size it for a peak you rarely hit, and you have bought an expensive idle. PTUs reward high, steady, predictable utilisation — and punish the opposite.

The choice between the two is arithmetic, not preference. Pay-as-you-go cost rises with every token; PTU cost is flat once bought. Below a certain steady volume, per-token is cheaper because you are not paying for idle capacity. Above it, reserved capacity is cheaper because per-token has no volume break and you are paying full rate on a lot of tokens. Where those lines cross is your decision point.

The crossover is driven by two variables: volume and steadiness. High volume alone is not enough — a workload that is high on average but wildly spiky may still favour pay-as-you-go, because PTUs sized for the peak sit idle in the troughs. It is sustained, predictable volume that tips the maths to PTUs: a production application serving steady traffic through the day, where the reserved capacity is genuinely used most of the time.

This is why you do not make the decision at pilot. At pilot, volume is low and unknown, so pay-as-you-go is right. You instrument the workload, learn the real token profile and the shape of demand, and revisit the choice when volume is high and steady enough that the PTU maths turns. Some estates end up hybrid — PTUs for the steady baseline, pay-as-you-go for the spikes above it — which is often the most economical answer of all.

Whichever billing model you are on, the cost is driven by tokens and by which model processes them — and those are levers you control. The first is context size. Every token you send in the prompt is paid for (on pay-as-you-go) or consumes reserved capacity (on PTUs), so sending a whole document when a retrieved paragraph would do is expensive. Tighten retrieval so the model gets the relevant chunk, not the whole knowledge base, and trim system prompts that have grown fat over time.

The second is model tiering. Not every task needs the frontier model. Routing easy, high-volume work — classification, extraction, short answers — to a smaller, cheaper model and reserving the larger model for the work that genuinely needs its reasoning can cut cost sharply, because the cheaper model is a fraction of the price per token. Match the model to the job rather than sending everything to the most capable, most expensive option.

The third is caching and output control. Cache and reuse responses where the same question recurs, so you are not paying to regenerate an identical answer. Cap output length — a model told to answer in a sentence generates far fewer (dearer) output tokens than one left to ramble. And control retries and agent loops, because an unbounded retry or a chatty multi-step agent can quietly multiply the token count of a single interaction. These levers apply on day one and matter more as volume grows.

Start on pay-as-you-go. At pilot and early rollout, volume is low and unpredictable, and per-token billing means you pay for exactly what you use while you learn the workload. Trying to size PTUs before you know your token profile is how you buy expensive idle capacity.

Instrument from the start. Track token usage per feature and per user, watch input versus output, and know your cost per interaction — this is the data that tells you both when to optimise and when the PTU maths turns. Apply the levers early: tight retrieval, model tiering, caching, output caps. They cost nothing to adopt and they lower the whole curve, which also pushes the PTU crossover further out.

Move to PTUs when volume is high and steady enough that reserved capacity is cheaper and the predictable bill and latency are worth the commitment — and consider a hybrid, PTUs for the steady baseline and pay-as-you-go for the spikes. The sequence is: pay-as-you-go and instrument, optimise the tokens and the model routing, then reserve capacity for the steady core once the numbers say so. Decide it on the data, not on a default.