Multi-turn protocol behavior evaluation on Protocol Agent v1
96.1Primitive Selectiondeepseek-v3p1-tuned
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| deepseek-v3p1-tunedtype=tuned2026.02 | 96.1 | 84.5 | 68.8 | 32.2 | 75.5 | 69.3 | |
| qwen3-30b-i2507-tunedtype=tuned2026.02 | 91.7 | 78.5 | 62.9 | 44.4 | 63.9 | 67.6 | |
| deepseek-r1-tunedtype=tuned2026.02 | 89.2 | 76.7 | 60.5 | 39.8 | 65 | 66.2 | |
| gpt-oss-120b-tuned1type=tuned2026.02 | 85.5 | 77.8 | 49.5 | 45.2 | 49.5 | 60.3 | |
| gpt-oss-120btype=base2026.02 | 81.8 | 72.8 | 43.5 | 50.2 | 42.5 | 58.2 | |
| deepseek-r1type=base2026.02 | 75.5 | 37.5 | 37.8 | 43.9 | 38.5 | 45.7 | |
| gpt-oss-20btype=base2026.02 | 74 | 58.3 | 34.7 | 68 | 30.7 | 53.1 | |
| deepseek-v3p1type=base2026.02 | 71.2 | 50.5 | 38.4 | 43.2 | 35.4 | 47.2 | |
| llama4mavericktype=base2026.02 | 65.9 | 31.1 | 28 | 50.3 | 20.5 | 38.8 | |
| qwen3-30b-i2507type=base2026.02 | 62.1 | 49.2 | 26.3 | 35.9 | 23.7 | 39 | |
| mistral-small-24b-i2501type=base2026.02 | 61.5 | 42 | 20 | 41.2 | 19.5 | 36.8 |