Provider Format Leak

Signs and symptoms

A back-office automation reconciles payments and issues invoices. The team wants to try another model, from another provider, on one of the steps. A code search shows the size of the problem (anonymized):

$ grep -rl "tool_use\|stop_reason\|content\[0\]" src/ | wc -l
31

src/billing/reconcile.py:   if resp.stop_reason == "tool_use":
src/billing/reconcile.py:       block = resp.content[0]
src/invoices/emit.py:       args = msg.content[1].input["tax_id"]
src/reports/daily.py:       history.append({"role": "assistant", "content": resp.content})
src/crm/sync.py:            if part["type"] == "tool_result" and part["is_error"]:
...

One provider’s message format (block types, field names, stop reasons) appears in 31 business logic files. Switching the model for one step means touching all of them.

On the NF-e issuer (Brazil’s electronic invoice) that hurts immediately: the module that builds the invoice XML reads resp.content[0].text, reconciliation reads choices[0].message, and the nightly job stores the API’s raw JSON in Postgres. Three ways to say “the model answered”, none of them yours.

Signs of a Provider Format Leak:

• Business code reads fields specific to an SDK’s response (content[0], stop_reason, choices[0].message).
• Tools return results in one provider’s block format.
• Conversation histories are stored in the database in one provider’s format.
• Tests depend on the raw response format.
• Switching models or providers is estimated in weeks.

flowchart LR
  P(["1. Provider SDK"]):::llm --> B1["2. reconcile.py"]:::tool
  P --> B2["2. emit.py"]:::tool
  P --> B3["2. sync.py"]:::tool
  P --> DB[("3. History in the provider's format")]:::store

1. A single SDK, with its message format.
2. Every business module reads that format directly.
3. Even stored data ends up tied to it.

Check +20 XP on first tryThe team wants to try a model from another provider on the reconciliation step only. The estimate is 'three weeks, 31 files'. What does that estimate expose first?

The two providers are too incompatible to live in the same product

The current provider's API format is spread through the business code

You lack a framework that already abstracts every provider

Reasons

• It starts with one provider. Using the SDK’s format directly is fastest. The first tutorial takes choices[0].message.content and moves on. Six months later, that is in billing, in the CRM, and in the NF-e job.
• Copying documentation examples. Sample code reads SDK fields right where the response is used. You paste the sample inside emit.py instead of behind a client.
• No adaptation layer. Nobody built a single place that converts the external format into the system’s own. Each module did its own read “just this once”.
• The database became the SDK’s attic. Storing the API JSON looks great for debug. A year of conversations becomes unreadable for another provider, and a field renamed in API v3 breaks test replay.

The trap: adopting a library that talks to several providers and, at the first difference, reading raw_response. The abstraction exists in package.json and died in an if.

Cost of ignoring

• Locked into one provider. Comparing or switching models becomes a project. An eval of a new model is not “point another key”: it is a refactor.
• Fragility to SDK updates. A format change breaks several modules at once. The provider’s changelog becomes an internal incident.
• No plan B. When the provider goes down, there is no quick way to divert to another. See Single Provider Dependency. The two smells travel together: that one is having no alternative, this one is the format that blocks the alternative.
• Hard tests. Simulating responses requires reproducing the provider’s full format, with blocks, indexes, and flags. A reconciliation test should not know what stop_reason is.

OpenAI uses choices[0].message and finish_reason. Anthropic uses a list of blocks in content and stop_reason. Google has another envelope. None of those names should appear in invoices/emit.py.

Treatment

Refactoring	Choose it when
Normalize Tool Output	The leak sits at the boundary with models and tools. A thin layer converts responses and results into your domain’s types, and only that layer knows the SDK.
Expose Tool via MCP	The tools are written in one provider’s format. Exposed through MCP (Model Context Protocol), they work with any client that speaks the protocol. Target: MCP Adapter.

The single client returns your types: TextTurn, ToolCall, ToolResult, Stop.completed | Stop.tool | Stop.filtered. History in the database stores those types. Each adapter (Anthropic, OpenAI, a third) translates at the edges.

function complete(req: DomainRequest) -> DomainResponse:
    # only this module imports the SDK
    raw = provider.sdk.messages(to_provider(req))
    return from_provider(raw)

The rest of the back office calls complete. Switching the reconciliation model is configuration plus one adapter, not 31 files.

This does not erase exclusive features. If you use a mode only one provider has, isolate that mode in a specific adapter and leave the rest of the domain unaware. Do not spread the exclusive field across five services “because it is already in the response”.

Check +20 XP on first tryA library already abstracts three providers. Even so, emit.py reads response.raw.stop_reason to decide whether to issue the invoice. What do you do?

Nothing: the library proves the team is not locked to one provider

Swap the library for another, hoping the field will come mapped

Expose the stop reason as a type of yours in the layer and ban raw in business code

Payoff

• The SDK appears in one module, not thirty.
• Trying another model on one step becomes a configuration change.
• Data stored in your own format, which survives a provider switch.
• Simpler tests that simulate responses in the domain’s format.

When to ignore

• A small script, one provider, no plan to switch. An adaptation layer is dead weight. If the script grows into a service, the layer enters on the first refactor.
• Provider-exclusive features. If you rely on a feature only one provider offers, accept the coupling at that point and isolate it, instead of hiding it behind a generic format that cannot represent it.

Is it this smell?

Check +20 XP on first tryCode calls the SDK in a single module, llm_client.py, which returns domain objects (Classification, ToolCall). Other modules only use those objects. Is this a Provider Format Leak?

Yes, the SDK is still in the code

No

Yes, until you stop using an SDK and speak HTTP by hand

Check +20 XP on first tryConversation history is stored in the database exactly as the API returns it, with the provider's block types. What is the risk?

None, it is just storage

Data tied to the provider's format

Only if the database has no JSON schema

Check +20 XP on first tryThe team uses a library that abstracts several providers. Even so, code reads one provider's fields through a raw_response. What is going on?

All good, there is an abstraction

The smell came back through a side door

It is Single Provider Dependency, not this smell

Check +20 XP on first tryA single module uses a provider-exclusive mode (a tool type the others do not have), behind an adapter. The rest of the system does not see the field. Is this a Provider Format Leak?

Yes, an exclusive feature is already a leak

Yes, because one day you will want another provider for that mode

No, the coupling is isolated at the point that represents it