Find the model and harness that make your product better.
Send us 200–500 of your real cases. We run them across models and harnesses, grade every answer with code, not another model, and show you which combination your users would notice. Where the agent moves money, every payment is checked against the ledger: the right party, the right amount, within its limits, and reported truthfully. Cost comes with it, not first. The report is one your model-risk team can file.
Winged Victory of Samothrace · Louvre · CC0the prow · slope 0.135 · 7.7°The report
- Cases
- 200–500 of yours
- Candidates
- Three to ten model and harness combinations
- Grading
- By code, not by another model; payments against the ledger
- Cycle
- Four weeks
- Output
- Which combination lifts quality, what it costs, and every case we could not grade
| Configuration | Task success | 95% confidence interval | Attempts per success | Cost / completed task | Meets the bar |
|---|---|---|---|---|---|
| Model A · harness 1 · retry ×2 | 93% | 90.1–95.1% | 1.4 | $0.046 | yes |
| Model B · harness 1 | 94% | 91.2–95.9% | 1.06 | $0.126 | yes |
| Model B · harness 2 | 98% | 96.1–99.0% | 1.1 | $0.141 | yes |
| Model C · harness 1 · 2 retries | 61% | 56.1–65.7% | 2.6 | $0.015 | no |
Illustrative example, 400 cases. A row meets the 90% bar only if the low end of its interval clears it. Model B · harness 2 is the only row clearly ahead, at three times the cost per completed task.
How it runs
200–500 cases.
Real inputs, real expected outcomes, under NDA. We write the grader with you.
Every model and harness combination.
Three to ten combinations, several runs each, retries counted, every call priced at what it actually cost, not list price.
Which combination your users would notice.
Task success per combination with confidence intervals, every payment the agent got wrong, cost alongside, and the cases we could not verify.
Nothing is trained on your data.
Our gateways store the prompts and completions of each call, with the paying wallet or account, in an access-restricted store used for abuse investigation and billing disputes; it is never sold, shared, or used for training. Providers are called on paid API tiers that do not train on API traffic.
Your cases stay under NDA.
We write the grader with you. Your cases are never sold and never go into anything we sell. Security