Human behavior simulation on SHOP-CART (test)
0.2772Exact Action AccuracyShop-R1
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Shop-R1Model=Qwen-2.5-3B-Instruct2025.07 | 0.2772 | 0.364 | 0.3128 | |
| Shop-R1Model=Qwen-2.5-0.5B-Instruct2025.07 | 0.2772 | 0.3183 | 0.212 | |
| Shop-R1Model=Qwen-2.5-1.5B-Instruct2025.07 | 0.2411 | 0.3454 | 0.2919 | |
| SFTModel=Qwen-2.5-3B-Instruct2025.07 | 0.1676 | 0.2225 | 0.2452 | |
| SFT + RL (Binary)Model=Qwen-2.5-3B-Instruct2025.07 | 0.1655 | 0.2374 | 0.2807 | |
| SFTModel=Qwen-2.5-1.5B-Instruct2025.07 | 0.1086 | 0.2358 | 0.2902 | |
| SFTModel=Qwen-2.5-0.5B-Instruct2025.07 | 0.099 | 0.1772 | 0.2161 | |
| Zero-shot promptingModel=Qwen-2.5-0.5B-Instruct2025.07 | 0.0676 | 0.1288 | 0.1555 | |
| RL (Binary)Model=Qwen-2.5-3B-Instruct2025.07 | 0.0101 | 0.0617 | 0.0992 | |
| Zero-shot promptingModel=Qwen-2.5-1.5B-Instruct2025.07 | 0.0053 | 0.0394 | 0.0616 | |
| Zero-shot promptingModel=Qwen-2.5-3B-Instruct2025.07 | 0.0032 | 0.1533 | 0.1615 |