Evals bench
Every model on every task: how often it is right, and what a right answer costs
Content checks, search, episode drafts and localisation on the same synthetic test set. Frontier, open models on Dvar and code-only routes side by side, ranked, then split by language and genre, down to each test case.
Internal benchmark on synthetic data · run run-202610230802 · 2026-10-23 · 7 model runs, same instruction per task
Head to head · Content checks
◆ Dvar tiered (code, open model, frontier when unsure)
90%
122/135 scenes right
₹0.10
per correct scene
claude-sonnet-5.5
87%
118/135 scenes right
₹0.35
per correct scene
3.4×
lower cost per correct scene on Dvar, at 90% right against 87%.
Correct means every problem in the scene caught, no false alarm. List prices.
4 ways to run it, ranked
Open model on Dvar, IndiaFrontier model| Model or route | Share right | Per scene | Per correct scene |
|---|---|---|---|
| 1Dvar tiered (code, open model, frontier when unsure) | 90% | ₹0.09 | ₹0.10 |
| 2Dvar · GLM 5.3 | 90% | ₹0.09 | ₹0.10 |
| 3claude-sonnet-5.5 | 87% | ₹0.31 | ₹0.35 |
| 4Dvar · gpt-oss-120b | 87% | ₹0.02 | ₹0.02 |
Correct means every problem in the scene caught, no false alarm. Open the Content checks tab →
By language
Share right per group. One average hides where a model is weak; this is the table a routing rule is written from.
| Group | claude-sonnet-5.5 | ◆ Dvar · gpt-oss-120b | ◆ Dvar · GLM 5.3 | ◆ Dvar tiered (code, open model, frontier when unsure) |
|---|---|---|---|---|
| English (26) | 92% | 96% | 96% | 96% |
| Hindi (72) | 79% | 85% | 88% | 88% |
| Hinglish (37) | 100% | 84% | 89% | 92% |
Test cases
| scene | Answer key | claude-sonnet-5.5 | Dvar · gpt-oss-120b | Dvar · GLM 5.3 | Dvar tiered |
|---|---|---|---|---|---|
| Haveli Ka Raaz · ep 55 | Mixed context | ✓ | ✓ | ✓ | ✓ |
| Contract Wali Shaadi · ep 19 | Clean | ✓ | ✓ | ✓ | ✓ |
| The Billionaire's Second Chance · ep 25 | Clean | ✕ | ✓ | ✓ | ✓ |
| Office Wala Pyaar · ep 17 | Foul language for the rating, Mixed language | ✓ | ✓ | ✓ | ✓ |
| Gaon Ki Chudail · ep 46 | Mixed context | ✓ | ✓ | ✓ | ✓ |
| Crime Files Lucknow · ep 17 | Clean | ✕ | ✓ | ✓ | ✓ |
| Maa Ka Aanchal · ep 55 | Foul language for the rating | ✓ | ✓ | ✕ | ✕ |
| Zero Se Crorepati · ep 24 | Mixed context | ✓ | ✓ | ✓ | ✓ |
| Haveli Ka Raaz · ep 52 | Generation residue | ✓ | ✓ | ✓ | ✓ |
| Contract Wali Shaadi · ep 61 | Clean | ✓ | ✓ | ✓ | ✓ |
| Badla Ek Aur Baar · ep 36 | Clean | ✓ | ✓ | ✓ | ✓ |
| The Billionaire's Second Chance · ep 36 | Clean | ✓ | ✓ | ✓ | ✓ |
| Office Wala Pyaar · ep 55 | Clean | ✓ | ✓ | ✓ | ✓ |
| Gaon Ki Chudail · ep 10 | Generation residue | ✓ | ✓ | ✓ | ✓ |
| Crime Files Lucknow · ep 11 | Clean | ✓ | ✓ | ✓ | ✓ |