Choose tasks, agents, treatments, and rubrics.
The easiest way to build evals and custom benchmarks for real-world agentic tasks
Run eval experiments at scale in realistic environments on fully managed cloud infrastructureDefine custom task sets to build your private benchmarks, measure how well agents can use any product, and find the best agent-model combinations for your use cases. Generate insights on the fly, such as product interface friction, token inefficiencies, performance differences, and so much more.
A video-style demonstration of building and launching an agent experiment in Oqoqo: describing a task, selecting tasks, configuring agents, treatments, runs and evals, then launching runs on sandboxed machines and reviewing pass and fail results with full step-by-step trajectories.
Configure an experiment: tasks, agents, treatments, rubrics, and headless interfaces.Configure an experiment: tasks, agents, treatments, rubrics, and headless interfaces.
This is a simplified, representative recreation. The full Oqoqo product is richer and more nuanced. Explore Oqoqo.
Evaluate any agent, any model, all at once
- Claude Code
- Codex
- Cursor
- GitHub Copilot
- OpenCode
- Grok Build
- OpenClaw
- Pi
- Hermes
- Antigravity
Managed infrastructure
Large experiments run quickly and reliably in the cloud on durable workflows and orchestration we manage.
Bring your own models
Connect the model providers and subscriptions you already use, then choose the model for each agent.
Compare treatments
A treatment is the package of tools associated with a product, such as skills, MCP servers, CLIs, and SDKs, so you can compare one package against another.
Rich customizable environments
Each task runs on its own sandboxed machine. Load optional repos and files, and customize the dependencies and tooling as needed.
What you can do with Oqoqo
Agents are becoming software's primary user.Build agent-first products any agent can use.
Point any agent at any product on any real-world agentic task. See every step it took, where it got stuck, and what fixing that is worth.
Define, run & analyze experiments from wherever you work
StripeAll experimentsNew experiment
task: product-pricetreatment: stripe-mcp
- Tasks
- product-price +2
- Agents
- Claude Code, Codex
- Treatments
- Raw agent · MCP
- Machine
- Stripe CLI sandbox
- Runs each
- 2
24 runs
Define the experiment
Choose tasks, agents, treatments, and rubrics.
StripeAll experimentsStripe interface evalsRuns
machine: Stripe CLI sandboxisolatedreproducible
sandbox-01done
sandbox-02done
sandbox-03running
runs18 / 24
Run each task in its own environment
Each run includes project state, context, and the tools the agent needs.
StripeAll experimentsStripe interface evalsTrajectory
product-priceFail
TrajectoryFrictionsOutput
#1
Instruction
Create a product named Pro plan with a $20 monthly price in test mode.
#2
Assistant
I will create the product, then attach a recurring price.
#3
Tool · bash
curl api.stripe.com/v1/prices -d interval=month
#4
Tool · bash
error: received unknown parameter: interval
Capture the full trajectory
See tool calls, commands, and where the agent stopped.
StripeAll experimentsStripe interface evalsCompare
Pass rate+25 pp lift
Raw agent33%
Stripe MCP58%
Run metrics
- Steps
- 172
- Tool calls
- 13
- Tokens
- 2,630k
- Frictions
- 3
- Duration
- 24m 00s
Measure the effect
Compare pass rates, tokens, and frictions.
StripeAll experimentsStripe interface evalsNew experiment
improve: headless interface
MCP server + skill updated
iteration 3
MCP · launch experimentready
CLI · oqoqo experiments runready
Fix and re-run
Fix what failed, then relaunch.
Build evals once and keep iterating to improve agents and agent-facing products
01
Input
Held constant across the grid
Evals
Files
Instructions
02
Sandbox
Isolated, reproducible runs
Agents
Treatments
Machine
Runs each
03
Output
Everything the run produced
Metrics
Trajectory
Diff
04
Loop
Ship the next interface change
Analyze
Improve
Rerun