Why the cheapest enterprise strategy is rarely all-API or all-on-premises, and how to draw the line between them.

Most enterprise AI decisions start with the wrong question. Teams ask whether to build on a frontier API or to self-host an open model, as if the two were mutually exclusive and one had to win outright. Framed that way, the answer is always unsatisfying. The API looks expensive at scale. On-premises looks risky and capital-heavy at the start. Both objections are real. Neither settles the matter, because the choice is not binary.

The right question is narrower and more useful. It is not which platform to pick. It is where each token should run. A frontier API and a self-hosted open model have opposite cost structures. Used together, and routed well, they cost far less than either used alone. This article sets out the economics behind that claim, shows where the break-even sits, and describes how a hybrid deployment is actually put together.

Two cost structures, not one price

The reason a hybrid design works is that the two options are not priced on the same axis. Comparing their headline rates tells you almost nothing. What matters is the shape of the bill as usage grows.

A frontier API is a pure variable cost. Opus 4.8, for example, is priced at $5 per million input tokens and $25 per million output tokens. There is no capital outlay. The bill scales linearly with use, from the first token to the last. That is the strength. You pay only for what you consume, and you can start today. It is also the weakness. The bill never stops, and it moves with things outside your control: usage spikes, priority-tier changes, tokeniser changes, and list-price revisions.

A self-hosted open model is the opposite. It is a fixed annual cost that barely moves with usage, capped by the throughput of the hardware. You pay for the node whether it runs at five per cent load or ninety-five per cent load. Once it is paid for, each additional token is close to free until the hardware saturates.

Put the two on the same chart and the shape of the decision is obvious. The API is a straight line from the origin. On-premises is a flat line set by the annual cost of the box. They cross at a single point. Below it the API is cheaper, because no capital sits idle. Above it on-premises is cheaper, and the gap widens with every additional token.

Figure 1. The API bill rises with every token. The on-premises node is a flat annual cost. They meet at roughly $135k a year, about 1.1 billion tokens a month at a 1:3 output-to-input mix. At four billion tokens a month the API costs around $480k a year, roughly 3.5 times the cost of the self-hosted node.

The fixed cost is not a guess. Take GLM-5.2 as the reference model. It is a 744-billion-parameter mixture-of-experts model with about 40 billion parameters active per token. The native FP8 checkpoint fits a single eight-GPU node, and it is released under the MIT licence, so fine-tuning, quantisation and full operational control are all permitted. On an eight-way AMD MI300X node the annual cost of ownership works out at roughly $135k: about $90k a year for the amortised hardware over a three-year refresh cycle, plus about $44k a year to power, cool and support it. Across the plausible range it is roughly $105k to $163k a year, and a four-year life pulls it towards $100k.

Where the line sits

The crossover is the point at which the annual API bill equals the node’s annual cost of ownership. In dollars it is unambiguous. If the frontier bill would run above about $135k a year, on-premises is cheaper. Below it, the API is cheaper, because no capital is stranded.

Two reference points make the line concrete. At a 1:3 output-to-input mix, typical of retrieval and agentic traffic:

$135k / year ≈ 1.1 billion tokens / month
280 million output plus 840 million input tokens a month costs about $135k a year on the API. This is the crossover, where the two options cost the same. Double the volume and the API bill doubles, while the node does not move.

The important feature of this number is how low it is. $135k a year sounds like a great deal of money until you translate it into users. In headcount terms it is only about 400 to 800 people on a frontier-backed assistant. Anything past a small departmental pilot crosses it.

Why enterprise scale clears the crossover

A worked example sets the scale. Take an internal knowledge assistant for 1,000 employees. Each runs 20 interactions on a working day. Each interaction carries about 8,000 input tokens and 800 output tokens, and there are 22 working days in the month. That is roughly 3.9 billion tokens a month.

On a frontier API, with no caching, that costs about $26k a month, or roughly $317k a year. With aggressive prompt caching at an 80 per cent hit rate it falls to about $14k a month, or roughly $165k a year. Both figures sit well above the $135k line, and this is a fairly modest deployment.

The ceiling then rises sharply with scale. The chart below places the common deployment sizes against the crossover. Note the horizontal axis is logarithmic, because the range runs across three orders of magnitude.

Figure 2. A departmental pilot sits below the line, where the API is the cheaper choice. Almost everything above a pilot clears it. Ordinary internal use runs 1.5 to 10 times the threshold. Customer-facing and agent-heavy use runs 10 to 100 times or more. Agentic traffic is the fastest-rising line: a single task can consume hundreds of thousands of tokens, and total agent consumption is projected to grow around 24-fold by 2030.

So for any enterprise at genuine scale, exceeding $135k a year on a frontier API is more likely than not. For ordinary internal use it is commonly 1.5 to 10 times the threshold. For customer-facing or agent-heavy use it is 10 to 100 times or more. A 60,000-user footprint of the kind already running in production would land near $10M to $19M a year if it could run on a frontier API at all, which for sovereignty reasons it often cannot.

What the market already shows

This is not a projection. It is visible in disclosed figures. The clearest numbers come from token-heavy coding tools, which lean towards the extreme end of usage, but the economics they reveal are general.

Cursor, the AI coding tool, was estimated by one investment firm to be paying about $650M a year to its model supplier against roughly $500M in revenue. That is a negative 30 per cent gross margin. The cost was also volatile. Its cloud bill doubled in a single month when a new priority tier launched. Its response was to build a proprietary model and route the bulk of traffic to cheaper models, after which it reached positive gross margins on enterprise sales.

The markup is structural, not incidental. A model maker pools and subsidises its heaviest users. An outside buyer pays per token at list price with the supplier’s margin included. One provider disclosed a single user consuming tens of thousands of dollars of usage on a $200-a-month plan. That gap is absorbed inside the platform. It is not available to a company buying tokens from the outside.

−30% gross margin
A leading coding tool was reportedly paying about $650M a year against $500M in revenue before it moved most traffic off the frontier model. Inference now consumes about 23 per cent of revenue at scaling-stage companies, roughly 30 points below normal software margins.

At the sector level, enterprise model-API spend doubled to about $8.4 billion in 2025. The recurring resolution, wherever volume is high, is the same. Move the bulk of traffic off the frontier API and reserve that API for the work only it can do.

The capability question

The obvious objection is capability. A frontier model is a frontier model for a reason. Does routing away from it mean settling for worse answers?

For most enterprise work, no. GLM-5.2 is the leading open-weights model. It sits a notch below the frontier on the hardest reasoning and longest-horizon agentic tasks. But the bulk of enterprise work is not that. It is retrieval, extraction, classification, summarisation, routine drafting and standard agentic flows. On this work an open model at this level is sufficient. The frontier edge only shows up on the hardest tail of tasks.

That is the whole basis for routing. Send the bulk to the open model and you remove most of the cost while losing little capability where it counts. Reserve the frontier API for the tail and you keep its advantage exactly where it earns its premium. You are not trading capability for cost. You are matching each task to the cheapest engine that can do it well.

How a hybrid deployment is built

In practice a hybrid deployment sits behind a single interface, so the split is invisible to the people using it. That layer is what Bud AI OS provides. It runs the open model on-premises, holds the frontier API in reserve, and presents both as one system. Requests from users, applications and agents arrive at it. Bud AI OS classifies each request, applies policy, checks the cache, and routes it to the right engine.

Figure 3. One system, two engines. The bulk of traffic, and all data bound by residency or sovereignty rules, goes to the self-hosted open model at a fixed cost. The small hard tail, on data that is permitted to leave the estate, goes to the frontier API and is charged per token. Responses return through Bud AI OS, so observability and guardrails apply consistently whichever engine served the request.

The bulk goes to the self-hosted open model. This path carries the high-volume work and, critically, all data that is bound by residency, sovereignty or air-gap rules. The data never leaves the estate. The cost is fixed and the latency is deterministic.

The hard tail goes to the frontier API. This path carries the small share of requests that genuinely need frontier reasoning, and only data that is permitted to leave the estate. The cost is per token, and it is incurred only on the fraction of traffic that actually reaches it.

Around both paths sit observability and guardrails. Logging, evaluation, cost accounting and safety checks are applied consistently, regardless of which model served the request. The result is one system with two engines, each doing the work it is best suited to. The routing layer is where the saving is realised, and it is what Bud AI OS is for.

The benefits that decide it

Cost is the headline, but three structural benefits sit outside the per-token comparison and often decide the matter on their own.

Data sovereignty and residency

For regulated, government and banking workloads, sending data to a third-party API is off the table for legal and contractual reasons. Here on-premises is not a preference. It is a requirement. And in that case it is also the cheaper option, so the decision makes itself. A hybrid design lets these workloads stay in-house without giving up frontier capability on the non-sensitive tail that is allowed to leave.

Cost predictability

A hybrid deployment replaces most of a volatile per-token bill with a fixed annual cost. What remains on the API is the small tail, so exposure to spikes, tier changes, tokeniser changes and price revisions shrinks with it. For budgeting and procurement, a number you can forecast a year out is worth a great deal. It turns an open-ended operating line into a planned capital and running cost.

Control and independence

The MIT licence permits fine-tuning and quantisation on the open model. Self-hosting gives deterministic latency, no rate limits, and no exposure to a vendor changing or deprecating a model underneath a live deployment. The frontier API remains available for the tail, but the core of the system is under your control. You decide when the model changes, not the supplier.

A framework for routing

The routing logic follows a short sequence of questions. The steps below run in order, and the first one that applies decides where the workload goes.

Figure 4. Sovereignty is settled first, before cost enters the picture. Volume and utilisation come next, because idle capital is pure waste. Only then is the remaining traffic split by difficulty. The bulk goes to the open model, and the hard tail goes to the frontier API.

First, sovereignty. If the workload is bound by residency, sovereignty or air-gap rules, it runs on-premises. This is settled before cost enters the picture.

Second, volume and utilisation. If sustained volume is comfortably above the $135k-a-year line and the hardware will run at high load, on-premises earns its place for the bulk. If the workload is a pilot, or bursty, or low-volume, the API is the safer call, because idle capital is pure waste.

Third, difficulty. Within the traffic that stays on the frontier-capable path, separate the bulk from the tail. Retrieval, extraction, drafting and standard agents go to the open model. The hardest reasoning and longest agentic chains go to the frontier API.

The honest caveats

The case is stronger for stating its limits plainly.

On-premises only reaches its low per-token cost at high utilisation. Right at the crossover the node is barely loaded, so its real cost per token is high and the API is the safer choice. Be clearly past the threshold before committing capital, not sitting on it.

On-premises carries a capital commitment, an operating burden, and hardware lead times of roughly eight to twenty-six weeks. The API is live today with none of these.

And the API side can be optimised. Routing routine traffic to cheaper tiers, and caching repeated context, can cut a frontier bill by two to ten times. The fair comparison is against a well-routed architecture, not against a naive frontier-on-everything rollout, which crosses $135k almost immediately. A hybrid design assumes this optimisation on both sides.

The bottom line

The financially optimal design is rarely pure API or pure on-premises. It is a routed mix. The self-hosted open model carries the high-volume, sovereignty-bound bulk at a fixed cost. The frontier API is held in reserve for the hard tail, where it earns its premium. This captures most of the saving of self-hosting while keeping frontier capability available where it is genuinely needed.

The practical requirement is the layer in the middle. A hybrid deployment is only as good as the routing, the observability and the guardrails that sit across both engines. That layer is Bud AI OS. It runs the open model on-premises, routes the hard tail to the frontier API, and applies one set of controls across both. Get that layer right and the choice between frontier and open models stops being a choice at all. It becomes a routing decision, made per request, on the merits.

In short

• The API is a variable cost. The self-hosted node is a fixed cost. They cross at about $135k a year, roughly 1.1 billion tokens a month.
• That line is low. It is only 400 to 800 users on a frontier-backed assistant, so most deployments past a pilot clear it.
• A modest 1,000-user assistant already runs $165k to $317k a year on the frontier API.
• Route the bulk to the open model and reserve the frontier API for the hard tail. This removes most of the cost and loses little capability.
• Sovereignty, cost predictability and control often decide the matter on their own, before unit cost is even counted.
• Bud AI OS is the layer that makes this work. It runs the open model on-premises, routes the hard tail to the frontier API, and applies one set of controls across both.

Figures in this article are illustrative. They assume Opus 4.8 list pricing of $5 per million input and $25 per million output tokens, a 1:3 output-to-input token mix, and a midpoint eight-way AMD MI300X node cost of ownership amortised over three years. A more output-heavy mix raises the crossover, and routing to cheaper tiers and caching reduce the API side and move the crossover outward. Replace the token volumes with a target deployment’s measured user count and per-task token profile for a firm figure.