We tested a popular vector database recently, and the eval came back positive, the agent got the right answer. Then we looked at the trajectory and realized the agent had never figured out how to use the product at all in fact it was thrashing and abandoning requests; it had fallen back to a SQLite solution and bypassed the product altogether. Nothing in that passing result would have told the team their product had just been routed around.
For the longest time, evals (or as we commonly call them, tests) focused on getting the right answer. Your unit tests expected assertions to be met, and as long as 2 + 2 came out to 4 it didn't much matter what route the code took to get there. Time and space complexity mattered, but not in our day-to-day sense of what a good outcome looked like. And where we did check the path, the mocks and spies asserting that a particular endpoint was called, we were checking a path we had already prescribed, because code was deterministic and the trajectory the developer chose was the trajectory that ran.
With agents, we see users carrying that same pattern forward and evaluating only the output. Of course they add variations to validate the result in different ways ("it must have said x," "it shouldn't have done y"), but many don't stop to realize that the trajectory matters as much as the outcome. Just because your MCP was available doesn't mean the agent used it, and just because the agent got the right answer doesn't mean your product helped it get there. We're dealing with a highly capable tool that can figure out its own way, and if you're building products that agents need to interact with, it's important to know whether an agent routes around you or actually leverages you.
This is where trajectory-based evals come in. Your evals need to look for criteria like "called bulk_insert() instead of looping single writes" or "loaded the schema-migration skill" and to distinguish between an agent that used your tool because it worked and one that used it, gave up partway, and finished the job some other way.
The trajectory also reveals the inefficiencies your product introduces, in tokens and in latency. Especially if you are a building block for other products, every inefficiency your headless interfaces introduce would multiply several times.