Customer Support
Benchmarks
TaskDatasetSOTA resultTrendResults
Customer Support Linear Pipeline S7: Triple Failure
9LLM Calls
3
Customer Support Linear Pipeline S6: Both Notif Down
8LLM Calls
3
Customer Support Linear Pipeline S5 Email Dies
5LLM Calls
3
Customer Support Linear Pipeline S4: Risk ($15k)
4LLM Calls
3
Customer Support Linear Pipeline S3: All Payment Down
0LLM Calls
3
Customer Support Linear Pipeline S2 Stripe Down
5LLM Calls
3
Customer Support Linear Pipeline S1: Happy Path
4LLM Calls
3