Instruction Following on 70 deterministic shopping tasks (live storefronts) (test)
11.4Avg Steps per RunGPT-OSS
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GPT-OSSNumber of Parameters=120B, Repetitions per task=102026.05 | 11.4 | 93.9 | 81.7 | 53.1 | 52.2 | |
| SIMPERSONATuning mode=Two-stage fine-tuning, Model Size=8x smaller than GPT-OSS, Repetitions per task=102026.05 | 14.9 | 77.7 | 91.4 | 76.9 | 59.3 |