Staying ahead of the cost-quality curve

A better model is not going to fix your agent.

If you’re running agents and have moved past your launch demo, you’re probably spending time evaluating models to find that perfect balance between cost and “quality”¹1however you choose to define it…. But models are capped at a quality ceiling: the maximum performance that can be eked with raw intelligence & prompt engineering alone. To break through the quality ceiling, you need to break down the task and run it in a harness that allows each step to be benchmarked, tested, and optimized.

The cost-quality curve

Assume you can simplify open benchmarks to a simple “cost per call” for a use case, and that you have a working quality measure on your task. Use these as axes, and every model you could run is a point on the curve. Roughly speaking they have a logarithmic relationship.

quality(model) ≈ k · ln(cost(model)) + c₀on one task

But this simple representation has a few problems:

1. you can’t rely on open benchmarks to measure model quality for your task,
2. how we choose a model is not merely checking the trendline for a “bargain”
3. model “bargains” are irrelevant if they fall below your “intent floor”

The intent floor is the quality below which our task output no longer does the job. If you ask for a blue button and get a gray one, the model is not doing what you need.

Internal benchmarks and the quality ceiling

Bad output doesn’t mean a bad model, but each one has limits. You can spend an eternity tweaking context management and engineering a prompt to squeeze a few more bips of performance out of a model, but there is an upper bound to any model’s raw capability. We call this the model’s “quality ceiling” — the maximum score it can reach on internal intent benchmarks.

Here I’m calling them “internal intent benchmarks” because we all know that public benchmarks are practically useless when it comes to evaluating how a model actually performs on your task. As your task gets more complex (i.e. the higher the “intent ceiling”), the more custom benchmarks are required, and the more time you spend hill-climbing performance with endless prompt & process tweaks. Eventually, every team realizes this is not scalable: if your agent depends too heavily on model capability and its behavior is a black-box, you’re quality-capped, period.

Breaking the curve

Raw model intelligence matters for complex reasoning but most agents rarely need it. Today, even the bottom end of the model curve can solve simple tasks with near 100% accuracy. Most tasks can be decomposed into primarily mechanical executions. A coding agent performs best as a multi-step process of research, planning, and review. The vast majority of its token use is file read and processing, which numerous researchers have shown can be accomplished with the simplest models. This means that at the extreme of decomposition, quality approaches its maximum and cost approaches its minimum.

quality(system) → bounded by the decomposition, not the modelandΣ cost(stepᵢ) < cost(one frontier call)

With full observability and routing control over every step, you can hold the intent floor as cheaply as possible while simultaneously breaking through the quality ceiling. Consider an agent as task → harness → result. At Sapiom we talk about converting “intent to outcome”; the task is the intent, the result is the outcome. So: the harness is the “how”. Sometimes, as engineers or builders, the “how” is clear — we define the steps, and allow the model reasoning to fill in the gaps. But those gaps are the model magic. We create the bounds, but we leave room for agentic creativity: the “how” is emergent. Based on its behavior, we improve, benchmark, and iterate for continual optimization past the quality ceiling.

Why the chart has to be redrawn

Of course, you must find the right balance between breaking apart tasks & setting guardrails, and allowing agents to use new capabilities and improved reasoning to self-optimize. Nothing above holds still. A new model ships every few weeks and lands somewhere on your curve, sometimes above the line. This is the importance of benchmarks, of A/B testing, and of constantly experimenting. It also helps that the vast majority of agentic tasks are fundamentally quality-capped. There are only so many quality improvements that one could make to a CRM intake agent, for instance. We give the agents room for creativity, and use their best ideas to continually improve their outcomes. Like moving from hand production to a factory floor, deeper understanding of the “how” allows us to perfect the result at an ever cheaper price.

The big deal

This isn’t a new concept: taking a big problem and breaking it down into small pieces is pretty much the entirety of software engineering. But for some reason the industry continues to make the mistake of expecting magic from models. It’s understandable — the way we do things has fundamentally changed, and LLMs still do feel like magic. Fortunately, they don’t have to be; stop optimizing your demo for prod, and break it up like any problem. You won’t lose the creativity, just the headache.

What about Jev?

Jev was released in the middle of writing this. Jev is an extremely quick and cost effective decision-making model, with many possible applications including rapid classification. The decomposition that we’re talking about above relies exactly on this kind of dead-simple, yes-no / true-false identification. You take a big task, discover the routes it can take, and route between paths while leaving the unknowns for the complex models. Over the next few days we will be exploring judgment and routing use cases for Jev within our benchmarking. Jev may drastically simplify the decomposition patterns that we’ve discussed above. Faster classification, better & cheaper decision routing — improving these are how we break through the quality ceiling while maximizing speed and cost efficiency.