Production
Benchmarks
TaskDatasetSOTA resultTrendResults
Production
0.029LIFT@30
12
Production-5k
0.638GAUC
6
Production WorkbackPlan (test)
79.8F1 Score
4
Production StatusReport (test)
79.4F1 Score
4
Production Email (test)
79.8F1 Score
4
Production DirectReports (test)
79.5F1 Score
4
Production UserManager (test)
79.7F1 Score
4
Production Calendar (test)
79.4F1 Score
4
Production WebSearch (test)
79.4F1 Score
4
Production (val)
97.6Accuracy
4
Production
91.3Mean Score
3
Production A/B experiment (904 queries)
0.368Sign Alignment
2
Production up to 21 January 2026 (deployment)
926Count
2