The True Cost of Switching AI Providers Mid-Project

Every six weeks, somebody on your team reads a benchmark, opens a pricing page, and says the same sentence: “We should switch to X, it is 40 percent cheaper.” Sometimes they are right. More often they have just signed your team up for two weeks of unbilled work that wipes out a year of the savings they were chasing.

Switching AI providers mid-project is not the same as switching cloud providers, payment processors, or even databases. The surface looks identical: same OpenAI-compatible endpoint, same JSON schema, same chat completion call. Underneath, the behavior of the model is the actual product, and that behavior is non-portable in ways nobody warns you about until you are already three days into the migration. This post is the warning.

Why the sticker price lies

Sticker prices on AI providers are quoted in dollars per million tokens. Claude Opus 4.7 sits around $15 input and $75 output per million as of mid-2026. GPT-5.5 is roughly $5 input and $20 output. Gemini 3.5 Flash hovers near $0.30 input and $2.50 output. DeepSeek V4 Pro undercuts everyone at $0.27 input and $1.10 output after their recent price cut.

Looking at those numbers, a CFO who has never read a prompt would tell you to migrate everything to DeepSeek tomorrow. The CFO would be wrong, and not by a small margin. The sticker price assumes the same prompt produces the same useful output across providers. It does not. The actual cost of a task is:

cost_per_task = price_per_token * tokens_needed * retries_until_correct

Two of those three variables change when you switch providers. Sometimes all three. A prompt that produces a clean answer in one shot on Claude Opus may need three retries, a longer system prompt, and a verification pass on a cheaper model. By the time you finish engineering around the gap, the cheaper model is no longer cheaper.

The five hidden costs of a migration

These are the buckets that never show up in the pricing comparison spreadsheet your team builds before pulling the trigger.

1. Prompt rewriting

Prompts are not portable. They look portable. You can paste your Claude system prompt into the GPT playground and it will run. It will also produce subtly worse output, and the worse part will not show up in your first ten test queries. It will show up in production three weeks later when a user reports a hallucination, a refusal, or a formatting break.

The reason is that every frontier lab trains its model on a different RLHF policy. Claude follows long detailed system prompts well and respects structured XML tags natively. GPT-5.5 prefers shorter directive prompts and ignores XML in favor of markdown. Gemini wants explicit step-by-step instructions and tends to be more literal. DeepSeek inherits a lot of GPT-style behavior but has its own refusal patterns around political content.

Real cost: one experienced prompt engineer, two to five days per major prompt, per target provider. If your project has 15 distinct prompts and you migrate to two providers in parallel for AB testing, that is 60 engineer-days. At a $200,000 fully-loaded annual cost, you just spent roughly $46,000 to find out which provider is cheaper.

2. Evaluation drift

Your eval suite was tuned against the source provider. The expected outputs in your golden dataset reflect the source model’s voice, structure, and tendencies. When you swap the model, half your assertions fail not because the new model is wrong but because it phrases things differently.

You now have two choices, both expensive. You can rewrite the eval suite to be more semantic and less literal, which takes weeks and pushes you toward LLM-as-judge evals that cost real money to run. Or you can declare the failures acceptable and accept that you no longer have a reliable regression net. The second option is what most teams actually do, and it is the most expensive choice of the three because it shows up six months later as silent quality degradation that you cannot trace back to the migration.

3. Tool-use and structured output incompatibilities

If your project uses function calling, JSON mode, or any structured output enforcement, you have provider-specific behavior baked into your code. Claude’s tool use returns a stop reason of tool_use with a structured input object. GPT returns tool_calls as an array with stringified JSON arguments. Gemini returns functionCall with a different schema entirely. DeepSeek mostly mimics GPT but has edge cases around nested objects.

You will write a shim. The shim will work for the common case. It will fail on retries, streaming, parallel tool calls, and any prompt where the model decides to emit content before or after the tool call. Each of those edge cases is an afternoon of debugging. The full shim, properly tested, takes one engineer one to two weeks.

4. Latency and throughput surprises

Pricing pages do not advertise latency, but latency is part of the cost. If your old provider gave you 80 tokens per second on streaming output and the new provider gives you 35, your user-facing chat feature just got noticeably worse. Your batch jobs got 2.3x slower. If you were running concurrent agent loops, you now hit rate limits twice as fast because each request stays open twice as long.

Rate limits themselves vary wildly. Anthropic tiers cap you at requests per minute and tokens per minute, with separate limits for each model. OpenAI uses TPM, RPM, and a daily token cap on lower tiers. Gemini caps you on QPM and has stricter content-type quotas. Moving a high-throughput workload across providers without renegotiating your tier means hitting walls you did not see in testing.

5. Reliability and contract risk

The provider you migrated from spent two years building track-record with your operations team. The new provider has not had its first three-hour outage with you yet. When it happens, you will not have runbooks, you will not have a contact at their SRE team, and you will not have alternate routing. The first outage of a new provider costs more than the first outage of an established one because everyone on your team is figuring it out in real time.

There is also contract risk. If you negotiated committed-use discounts with your old provider, you may be paying for tokens you no longer use. Most enterprise AI contracts in 2026 still come with annual commits in the six-figure range. Walking away from one of those mid-term is a financial decision, not a technical one.

A worked example

Take a realistic mid-sized project: a coding assistant generating 200 million tokens per month, split roughly 70 percent input and 30 percent output. On Claude Opus 4.7, that bill is around $9,600 per month. Migrating to GPT-5.5 at sticker price drops it to about $1,900 per month. Savings: $7,700 per month, or $92,000 per year.

Sounds obvious. Now layer in the migration cost.

Prompt rewriting across 12 prompts: $25,000 of engineer time spread over a month. Eval suite rebuild: $18,000. Tool-use shim plus regression testing: $12,000. Latency mitigation, fallback routing, runbook updates: $8,000. Total one-time migration cost: $63,000.

Then layer in the recurring overhead. The new provider needed 18 percent more tokens to produce equivalent output quality because the prompts are now longer and retry rates are slightly higher. That eats $340 per month of the savings. Latency forced you to add a second small-model verifier on top, adding $200 per month. Net real savings: roughly $7,160 per month against a one-time cost of $63,000. Payback period: nine months.

Nine months is acceptable if you are confident the project will run for at least 18. It is unacceptable if there is a meaningful chance you migrate again, the new provider raises prices, or the project gets cut. Most internal AI projects in 2026 are within all three of those risk envelopes.

When switching is actually worth it

It is worth it in three specific cases.

First, when the source provider has materially changed the terms of the relationship. Rate limits cut in half, pricing doubled, model deprecated, region restrictions imposed. In these cases the cost of staying is the new variable, and the comparison is forced.

Second, when you are at the start of a new project, not mid-project. Migration cost during greenfield is near zero because you have no prompts, no evals, and no shim to rewrite. Pick the cheapest provider that meets your quality bar and move on.

Third, when the gap is huge and durable. A 5x cost difference between providers that has held for at least two quarters is worth migrating for. A 1.5x difference is not, because that gap will probably close within six months and your migration cost will not amortize before it does.

If you are not in one of those three cases, the answer is to stay and either renegotiate or route incrementally.

Routing instead of switching

The smart play in 2026 is multi-provider routing, not migration. Keep your existing provider as the default. Add a cheaper or faster provider as a route for specific task types. Common splits that work:

• Cheap model for classification, intent detection, and simple summarization. Expensive model for synthesis, reasoning, and customer-facing writing.
• Local or open-weights model for high-volume internal tools where quality drops of 5 to 10 percent are acceptable.
• A second frontier provider as failover for outages and as a sanity check for high-stakes outputs.

This approach gets you 60 to 80 percent of the cost savings of a full migration with maybe 20 percent of the migration cost. It also de-risks you against any single provider going off-policy with pricing or limits. The work is in building a router that can carry prompt translation and structured-output normalization at the edge, which is roughly two engineer-weeks of one-time work and pays back forever.

This is exactly the bookkeeping problem tokenkarma is built to surface. When you can see, per task type per provider, what each query actually costs in dollars and in retries, the migration decision stops being a benchmark argument and becomes an arithmetic one. The teams that get this right are not the ones that pick the cheapest provider. They are the ones that know, for each kind of work, which provider is cheapest including all the overhead, and only re-evaluate that ranking when something meaningful changes.

The checklist before you migrate

If you are still convinced switching is the right move, run this list before you commit:

1. Quantify the sticker savings over a 12-month horizon, not a monthly one.
2. Estimate engineer-days for prompt rewriting, eval rebuild, and shim work. Multiply by 1.5.
3. Measure quality on a held-out eval set with at least 200 examples per task type. Look at the tail, not the mean.
4. Test latency and rate limits at 80 percent of your peak production load, not at idle.
5. Confirm the new provider has a roadmap that does not deprecate models you depend on within 12 months.
6. Negotiate exit terms with the current provider before migrating, not after.
7. Plan the migration as a route shift, not a switch. Run both providers in parallel for at least four weeks.

If those numbers still favor switching, do it. If they do not, the cheapest move is to stay, renegotiate, and route incrementally. The companies spending the least on AI in 2026 are not the ones running on the cheapest provider. They are the ones who stopped thinking of provider choice as a binary and started treating it as a per-task allocation problem.

The sticker price was never the bill. The bill is the sticker price plus everything you had to do to make the new provider behave like the old one. Until you have that number, you do not actually know which provider is cheaper.