Workato Model Gateway

Route every model call - from every provider - through one governed endpoint. Swap models, cap spend, and screen every prompt without touching a line of application code.
AI Gateway dashboard

Skipping a model layer is costing you

  • Workato icon — Revenue operations

    Cost

    Token spend arrives as one invoice with no team attribution. Everyone defaults to the frontier model. Reasoning models cost multiples of a standard call, untracked. One looping job can burn a month of budget overnight.

  • Workato icon — Error

    Reliability

    A deprecated model means a code change in ten services. A provider outage takes production down with no fallback. Provider lock-in — nothing routes around a bad vendor.

  • Workato icon — Lock

    Security

    PII goes to third-party providers because masking was never anybody's job. Prompt injection and jailbreak attempts arrive unscreened.

  • Workato icon — Compliance

    Governance

    Provider keys are handed out directly — nothing to revoke, cap, or audit. No record of which prompt hit which model. Every team writes its own retry, fallback, and logging code.

  • One change: swap the base URL and the key
    Get Started

    One change: swap the base URL and the key

    Point your application, agent, or AI client at the Workato gateway endpoint instead of the provider endpoint, and replace the provider API key with a Workato virtual key. No SDK change, no rewrite, no re-architecture. From that call onward, traffic is authenticated, routed by policy, screened, metered, logged, and attributed.

    Schedule a demo

    • Key Components

    Five Components, One Gateway

  • Governance

    Governance

    Issue a Workato credential that replaces your provider key in every application, carry its own routing rules, limits, budgets, and policies, revoke it without touching the provider account. Log every request and response with cost attributed by key, team, model, provider, and tag.

  • Intelligent and conditional routing

    Intelligent and conditional routing

    Map every request to a model and provider by virtual key, team, or tag, and set a workspace default with overrides. Block anything off the approved-model list in enforce or observe-only mode, and change the rules without an application deploy.

  • Reliability

    Reliability

    Send one request format across every provider, and automatically retry or fall back to an alternate model or provider when one degrades or errors.

  • Guardrails

    Guardrails

    Detect and mask PII before a request leaves the gateway, screen for prompt injection and jailbreaks on the way in, moderate content in both directions, and check grounding on responses that were given context to work from. Each guardrail runs in enforce or observe-only mode.

  • Cost control

    Cost control

    Normalize token accounting per request across every provider, split by input, output, reasoning, and cached tokens. Set token-per-minute rate limits and monthly budgets per key, with alerts before you hit the ceiling, not after.

    • Who it's for

    What each team can do with it

    • IT / Platform / AI Eng
      Give every team one endpoint for models, and change which model they get by config instead of by release.
      OpenAI · Anthropic · AWS Bedrock · Azure OpenAI · Gemini · OpenAI-compatible · Anthropic-compatible · BYOLLM
    • Application Engineering
      Get retry, fallback, and provider abstraction without writing any of it, and keep your existing SDK.
      OpenAI-compatible SDKs · SSE streaming · virtual keys
    • Security & Compliance
      Stop PII reaching a model provider, screen prompts for injection, and log every call.
    • Finance / FinOps
      See token spend by team, key, and model — including reasoning tokens separately — and cap it.
      Token and cost dashboards · CSV export · BI tools
    • Data Science / ML
      Compare models on real traffic and move production to a cheaper one without touching application code.
      Routing rules · approved-model list · self-hosted and BYO models

    IT / Platform / AI Eng

    IT / Platform / AI Eng
    How it works

    Model choice stops being a code decision

    Swapping a model, adding a provider, or blocking one becomes a configuration change instead of a release across every service that calls it.

    Works with

    One multi-provider LLM API for every model

    AI GW — provider strip

    Frequently asked questions

    • What is a Model Gateway, and how is it different from Workato's AI Gateway?

      The Model Gateway is the model layer specifically: one endpoint in front of every model provider, governing which models can be called, by whom, under what policy, at what cost. It's one of four surfaces inside Workato's AI Gateway — alongside the MCP Gateway, Agent Gateway, and API Gateway — all sharing one identity model, one policy model, and one audit trail. If you only need to govern model calls today, start here; the rest are there when you need them, without a second setup.

    • What's the difference between a Model Gateway and an LLM Gateway?

      Nothing meaningful — the terms are used interchangeably in the market, with "LLM gateway" more common among developers focused on routing across providers and "model gateway" more common where the emphasis is production governance. The Workato Model Gateway is both: routing, failover, token accounting, prompt and response screening.

    • How is a Model Gateway different from an API Gateway?

      An API gateway routes traffic and treats the payload as opaque — it knows the request arrived and where to send it. A model gateway reads what's inside: which model is being called, how many tokens it will cost, whether the prompt contains PII or an injection attempt. The difference shows up in the unit of control — API gateways enforce requests per second; model gateways enforce tokens and dollars per team, and can rewrite a request before it leaves. Most enterprises run both.

    • Does the Workato Model Gateway work with Claude, GPT, and Gemini?

      Yes — OpenAI, Anthropic, AWS Bedrock, Google Gemini, and Azure OpenAI, plus any OpenAI-compatible or Anthropic-compatible endpoint and self-hosted models you bring yourself. The gateway presents an OpenAI-compatible API, so existing SDKs work unmodified.

    • How do I switch models without changing application code?

      Model selection lives in a routing rule, not your application. A rule maps a request — by virtual key, team, or tag — to the model and provider it must use, with a default for everything unmatched. Change the rule and the next request goes to the new model; no deploy, no release. An approved-model list can run in observe-only mode first, so you see what would have been blocked before you block it.

    • Can I use my own self-hosted or fine-tuned model?

      Yes. Any endpoint that speaks an OpenAI-compatible or Anthropic-compatible API can be registered as a provider and governed by the same routing rules, limits, guardrails, and logging as a commercial provider.

    • Does the Model Gateway add latency to model calls?

      It adds a policy evaluation step on the request path, designed to be a small fraction of model inference time, which dominates end-to-end latency in practice. Policy decisions evaluate against cached configuration rather than a round trip per request.

    • Do I have to route all my traffic through it?

      No. Start with one application or one team, and expand when it's earned it. Adoption is a base-URL and key change, and so is backing it out.

    • How does the Model Gateway stop PII from reaching a model provider?

      PII detection runs on the outbound request inside the gateway, before it leaves for the provider, and masks what it finds. Where masking is a mandatory policy and the check can't run, the request is blocked rather than sent through unscreened. The audit record shows that PII was detected and which categories — never the values.

    • What audit trail does the Model Gateway produce?

      Every call is recorded: who invoked it, which model and provider it resolved to, which policies applied and what they decided, and what it cost in tokens. Records can stream to your SIEM or object storage — the evidence base EU AI Act readiness, SOX, and HIPAA programs ask for.

    • Can I see AI spend by team?

      Yes. Tokens are metered per request and normalized across providers, then attributed by virtual key, team, model, provider, and tag — with reasoning tokens broken out separately, since they cost materially more. Budgets and per-minute limits sit on the same numbers, with alerting before the ceiling.