MLPerf Agentic Inference Benchmark is the first of its kind to measure agentic AI performance across real enterprise business processes. Workato’s AI lab co-built the enterprise workflow workload together with AMD, Intel, NVIDIA, and MLCommons.
This week, MLCommons added a new track to MLPerf Inference, the industry’s most widely used standard for measuring how fast AI infrastructure systems actually perform. The MLPerf Agentic Inference Benchmark measures how efficiently LLM systems hold up under multi-turn agentic workloads, where context keeps growing and every turn depends on the one before it. This is the first MLPerf Agentic Benchmark measuring infrastructure performance on the complexity of real enterprise business processes.
The research community had no shared way to measure agentic efficiency, because it had no realistic workflows.
For years, LLM inference benchmarks measured one prompt in, one answer out: single-turn text generation, question answering, summarization. Those workloads still matter, but they focus on tasks, and they do not inform how efficiently a serving stack executes an agentic task: where agents that actually run in production are expected to handle end-to-end business processes across dozens of systems, data sources, and approvals.
A real agent doesn’t answer once and stops. A workflow agent gathers customer information, calls tools, interprets what comes back, asks for a follow-up, and keeps going until the task is done. The unit of work isn’t a request or task. It’s a trajectory towards an outcome: a sequence of dependent turns, where each turn carries the whole conversation towards a business metric or KPI. Context grows across the trajectory, so prefill and KV-cache pressure climb over time. Cache reuse stops being a side effect and becomes the core optimization. And because turns depend on each other, efficiency is no longer about how many independent requests you can fire; it’s about how much progress you can make for real users.
Frontier labs measure some of this internally; that’s part of why their agents are as good as they are. But they do it against private workloads in their AI lab, and the open research community had no shared benchmark to test performance against real enterprise processes. The reason wasn’t the metrics or the harness; those are tractable. The blocker is the workload itself.
And here the distinction matters. To measure agentic inference, you need a realistic workflow: the tools, business rules, task structures, and prompt shapes that determine how context grows and where a serving stack actually gets stressed in a real organization. You do not need the real traffic that runs through it, the actual customer requests and data. Traffic can be simulated, but a realistic workflow has to come from somewhere that actually runs enterprise agents. That separation is what makes an open benchmark possible at all. Enterprise workflows are the scarce, valuable part, and they’re exactly what generic or scraped data fails to capture, while the traffic over them can be generated synthetically, with no private data ever leaving the building. So the missing piece was not live traffic. It was a representative agentic workflow that someone could publish.
Why Workato: massive enterprise workflows, simulated data
Workato is the Enterprise Control and Execution Plane for AI with a mature enterprise MCP layer and a deep catalog of connectors and tools. For over a decade, Workato has orchestrated complex agentic workflows and mission critical business processes across live business systems. That gives Workato direct, firsthand knowledge of what enterprise agent workflows actually look like, and therefore what the traffic over them looks like.
An enterprise AI agent executing a business workflow starts every trajectory with a very large shared prompt: dozens of tool definitions, business rules, policies, and account context. It often resolves a case in relatively few turns, so the defining stress isn’t trajectory depth. It’s the enormous common prefix repeated across thousands of conversations, which makes prefix reuse and routing locality central to whether a serving stack is efficient or wasteful. The benchmark also includes an agentic coding workload, built from SWE-bench-style software tasks, whose deep and growing context stresses KV-cache capacity and long-context scheduling. Covering both gives the benchmark a wider range of production serving patterns than either alone would.
Open, and built with AMD, Intel, NVIDIA, and MLCommons through a new kind of collaboration.
The Workato AI Lab actively embraces open source. The traces and reference implementation are released under an open license, so any builder can pull the workload, benchmark their own serving stack against it, and contribute back. The more teams build, measure, and share against a common, enterprise-grade workload, the faster the whole ecosystem gets at running real agents.
The collaboration model itself is worth noting: this MLPerf Inference working group task force effort was chaired by Harshil Vagadia at NVIDIA and Tianmu Li at Intel, along with leading contributions from Poovaiah Palangappa at AMD as well as Rita Brugarolas Brufau at AMD; the coding trace generation and the Eagle3 speculative decoding head for Kimi-K2.6 (developed using NVIDIA ModelOpt) are contributed by NVIDIA; and on the Workato side, the workflow workload was built by AI researchers Zhaozhuo Xu and Qi Yang Huang, with the broader Workato AI Lab team. An enterprise agentic company contributed the workload that only it can produce, working alongside ML-systems and GPU-cloud partners who know how to measure and serve it. The combined benchmark is substantial: more than 900 multi-turn trajectories and more than 30,000 client-issued turns across the two domains, evaluated on long-context, thinking-capable models (Kimi K2.6 and Qwen3.6-35B-A3B) at up to 262K tokens of context, scored on a Pareto curve of aggregate throughput against per-user progress, with accuracy enforced at three levels so a system can’t win by quietly truncating answers. The benchmark lives inside the MLPerf Endpoints framework under MLCommons.
That’s the same conviction behind how Workato builds. Enterprises don’t transform because a model can reason; they transform when agents can execute reliably across real systems. Making that execution measurable, at the infrastructure level and with the rest of the industry, is part of making it dependable.ency is no longer about how many independent requests you can fire; it’s about how much progress you can make for real users.
Frontier labs measure some of this internally; that’s part of why their agents are as good as they are. But they do it against private workloads in their AI lab, and the open research community had no shared benchmark to test performance against real enterprise processes. The reason wasn’t the metrics or the harness; those are tractable. The blocker is the workload itself.
And here the distinction matters. To measure agentic inference, you need a realistic workflow: the tools, business rules, task structures, and prompt shapes that determine how context grows and where a serving stack actually gets stressed in a real organization. You do not need the real traffic that runs through it, the actual customer requests and data. Traffic can be simulated, but a realistic workflow has to come from somewhere that actually runs enterprise agents. That separation is what makes an open benchmark possible at all. Enterprise workflows are the scarce, valuable part, and they’re exactly what generic or scraped data fails to capture, while the traffic over them can be generated synthetically, with no private data ever leaving the building. So the missing piece was not live traffic. It was a representative agentic workflow that someone could publish.
Why Workato: massive enterprise workflows, simulated data
Workato is the Enterprise Control and Execution Plane for AI with a mature enterprise MCP layer and a deep catalog of connectors and tools. For over a decade, Workato has orchestrated complex agentic workflows and mission critical business processes across live business systems. That gives Workato direct, firsthand knowledge of what enterprise agent workflows actually look like, and therefore what the traffic over them looks like.
An enterprise AI agent executing a business workflow starts every trajectory with a very large shared prompt: dozens of tool definitions, business rules, policies, and account context. It often resolves a case in relatively few turns, so the defining stress isn’t trajectory depth. It’s the enormous common prefix repeated across thousands of conversations, which makes prefix reuse and routing locality central to whether a serving stack is efficient or wasteful. The benchmark also includes an agentic coding workload, built from SWE-bench-style software tasks, whose deep and growing context stresses KV-cache capacity and long-context scheduling. Covering both gives the benchmark a wider range of production serving patterns than either alone would.
Open, and built with AMD, Intel, NVIDIA, and MLCommons through a new kind of collaboration.
The Workato AI Lab actively embraces open source. The traces and reference implementation are released under an open license, so any builder can pull the workload, benchmark their own serving stack against it, and contribute back. The more teams build, measure, and share against a common, enterprise-grade workload, the faster the whole ecosystem gets at running real agents.
The collaboration model itself is worth noting: this MLPerf Inference working group task force effort was chaired by Harshil Vagadia at NVIDIA and Tianmu Li at Intel, along with leading contributions from Zhihan Jiang at NVIDIA, Poovaiah Palangappa at AMD as well as Rita Brugarolas Brufau at AMD; the coding workload and the speculative-decoding work came from the NVIDIA side; and on the Workato side, the workflow workload was built by AI researchers Zhaozhuo Xu and Qi Yang Huang, with the broader Workato AI Lab team. An enterprise agentic company contributed the workload that only it can produce, working alongside ML-systems and GPU-cloud partners who know how to measure and serve it. The combined benchmark is substantial: 990 multi-turn trajectories and more than 30,000 client-issued turns across the two domains, evaluated on long-context, thinking-capable models (Kimi K2.6 and Qwen3.6-35B-A3B) at up to 262K tokens of context, scored on a Pareto curve of aggregate throughput against per-user progress, with accuracy enforced at three levels so a system can’t win by quietly truncating answers. The benchmark lives inside the MLPerf Endpoints framework under MLCommons.
That’s the same conviction behind how Workato builds. Enterprises don’t transform because a model can reason; they transform when agents can execute reliably across real systems. Making that execution measurable, at the infrastructure level and with the rest of the industry, is part of making it dependable.