16External models

Hosted models integrated, and their trade-offs stated.

Integrating external models into your systems: the Gemini, Claude and ChatGPT APIs, or open-weight models hosted elsewhere. Supplied with them are the access layer, the spend controls and the fallback path — the parts that determine whether the integration holds when the provider fails.

When this is the right answer

A hosted model fits more often than the rest of this site implies. The conditions: the task is general rather than specific to your data; what you send is not sensitive; being in production this month outranks being independent next year. The trade is two-sided. Capability immediately, in exchange for data leaving the perimeter, a bill that scales with use, latency that depends on an outside network, and a version that can change outside your release cycle. Those are parameters for a decision, not arguments against one.

Your system
the calling code
the tasks worth sending out
Access layer
keys · limits · accounting
one place holding the keys
Redact
what must not leave
a boundary you control
Provider
the external model
the integration itself
Fallback
when it is not there
a defined answer when it is down
Nothing calls the provider directly. Everything a third party sees passes one place you control.

What it covers

  • Selecting the tasks a general model is applicable to
  • An access layer: keys, limits and per-team accounting
  • Redaction of fields that must not leave the perimeter
  • A fallback path for provider degradation or outage
  • Retry and timeout policy that does not amplify an outage
  • Response caching for repeated queries
  • Prompt and model version pinning, to track provider-side change
  • Integration into your existing services
  • Cost monitoring against a budget you set

You get

An integration with a single point holding the keys, a single boundary for data leaving, per-team attribution of the bill, and defined behaviour for the period a provider is unavailable.

Typically e-commerce and media teams with a general language task and no regulatory bar on sending it out, where reaching production this quarter outranks independence next year.

Book an architecture review
The cost of this option

Sending data to a third party is the cost of this option, not a detail of it. We build the integration, make the boundary for outgoing data explicit and controllable, and state which tasks the option does not suit. Where data may not leave the perimeter, this is not the service to choose.

Related services

Book an architecture review

← Back to balticdevops.io