One endpoint for every model, with policy on every call.
The AI gateway exposes an OpenAI-compatible API inside your network. Existing SDKs and tools point at it with a base URL change. Behind it, every request is authenticated, checked against policy and quotas, routed to an approved model, and logged.
Change the base URL. Keep your code.
The gateway speaks the OpenAI chat completions, embeddings, and models endpoints. Credentials are issued per application or user by Data.co Control, not shared provider keys.
import os
from openai import OpenAI
client = OpenAI(
base_url="https://ai.internal.example.com/v1",
api_key=os.environ["DATACO_GATEWAY_TOKEN"],
)
resp = client.chat.completions.create(
model="general-chat", # registry alias, not a vendor model
messages=[
{"role": "system", "content": "Summarize for a claims reviewer."},
{"role": "user", "content": claim_text},
],
)
print(resp.choices[0].message.content)Illustrative example · final API surface documented in /docs
What the gateway does before a model sees a token.
Identity
Per-user, per-application, and per-agent tokens tied to your identity provider; no shared keys.
Policy
The policy engine decides whether this identity may send this data to this model for this purpose.
Data loss prevention
Detectors for sensitive data can block, redact, or require approval before a prompt is forwarded.
Quotas
Token and request limits per user, team, or application, so one batch job cannot starve interactive users.
Routing
Aliases resolve to approved model versions, with fallbacks and separate pools for interactive and batch traffic.
Audit
Identity, alias, resolved model, decision, and token counts recorded for every call.
Commercial APIs are off until a policy turns them on.
Some teams want a hosted frontier model for specific, low-sensitivity tasks. The gateway lets you allow that narrowly instead of all-or-nothing.
| Control | Default | What you can configure |
|---|---|---|
| External provider access | Disabled | Enable per provider, per alias |
| Who may use it | Nobody | Specific groups, applications, or agents |
| What data may go | None | Only listed classifications; others blocked at the gateway |
| Logging | Always on | Retention and export to your SIEM |
Shadow AI is mostly a missing paved road.
Teams reach for personal accounts and browser extensions when there is no sanctioned way to use AI. Blocking domains only moves the problem. A gateway gives developers and business users an endpoint that works with the tools they already know, and gives security a single place to see and govern usage.
Because the gateway runs inside your network, it also works where outbound internet access is restricted or absent, including air-gapped environments.
The order of operations inside the gateway.
- 1
Authenticate
Resolve the bearer token to an identity with its tags, groups, roles, and declared scope. Suspended identities and revoked tokens stop here.
- 2
Normalize
Build one policy request for invoking the model endpoint, plus one read request per dataset the caller names, each carrying catalog classifications.
- 3
Evaluate
Run every request through the locally compiled policy set. If the agent acts for a user, it is evaluated as both and the stricter result wins.
- 4
Record
Append each evaluation to the audit log with subject, resource, decision, and policy IDs. Prompt text is never written to the log.
- 5
Combine and forward
Any deny means deny; otherwise any approval requirement means wait. Only an allow is proxied upstream, and streaming responses pass through.
Responses your client code should handle.
Every response carries a request ID, and policy outcomes add the decision and matching policy IDs as headers, so support tickets can reference an exact decision.
| Status | Meaning | What the client should do |
|---|---|---|
| 200 | Allowed and served | Use the response; streaming works as with any OpenAI-compatible API |
| 403 policy_denied | A policy refused this identity, model, or data | Show the user a clear message; do not retry automatically |
| 404 | Model alias not registered or disabled | Check the alias name against the registry |
| 409 approval_required | A policy requires a named approver | Surface the approval ID; the identical request succeeds once after approval |
| 502 | Allowed, but the model endpoint could not serve it | Retry with backoff; check model health |
| 503 | License expired past its grace period | Escalate to the platform team |
Questions buyers ask
Which OpenAI API endpoints are supported?
Chat completions, embeddings, and model listing, including streaming responses. Features that depend on a specific hosted provider may not be available for every private model; the documentation lists current coverage.
Will tools like coding assistants or chat front-ends work with it?
Many tools that let you set an OpenAI-compatible base URL and API key work without modification. Each tool's requests are then subject to the same identity, policy, and logging as any other caller.
How do quotas work?
Limits can be set on tokens and requests per user, team, application, or agent over a time window. When a limit is reached, the gateway returns a clear error rather than silently degrading other users.
Can the gateway redact sensitive data before sending to an external model?
Yes, if a policy allows an external model at all. Data loss prevention rules can redact, block, or require approval based on what is detected in the prompt.
Does the gateway add much latency?
The checks run locally next to your models, so overhead is small relative to model generation time. It depends on the policies and detectors you enable, and is visible in monitoring.
How does the gateway know which data is in a prompt?
Callers name the datasets in play with a request header or a field in the body, which is stripped before the request reaches the model. The gateway attaches each dataset's classification from your catalog, and DLP detectors can inspect the prompt itself where enabled.
Does the audit log contain our prompts?
No. Audit events are content-free: identity, model, datasets, decision, policy IDs, and explanation, never prompt text or completions. Records are hash-chained, so editing, deleting, or reordering an entry breaks verification.
How are credentials for an approved external provider stored?
The endpoint references a secret by name, and the gateway attaches it as a bearer token only when forwarding an allowed request. Callers never see the provider key, and external endpoints are disabled automatically in air-gapped mode.
See it against your own use case.
A 30-minute technical walkthrough, policy simulation on your scenarios, and a deployment recommendation for on-prem, private cloud, or hybrid.