Simulation Performance on R2
46AccuracyCAPO-tol1
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CAPO-tol1Model=Llama3.2, Setting=CAPO-tol12026.05 | 46 | 1.47 | 0.293 | 0.889 | 0.582 | 0.187 | 98.8 | 98.5 | 0.995 | 53 | — | |
| CAPOBackbone=Llama3.22026.05 | 46 | 1.47 | — | — | — | 0.187 | — | 98.5 | 0.995 | 53 | 0.293 | |
| CAPO-tol1Model=Qwen3, Setting=CAPO-tol12026.05 | 45.5 | 1.49 | 0.316 | 0.892 | 0.595 | 0.112 | 97.1 | 96.4 | 0.973 | 56 | — | |
| CAPOBackbone=Qwen32026.05 | 45.5 | 1.49 | — | — | — | 0.112 | — | 96.4 | 0.973 | 56 | 0.316 | |
| SFT-ind.Model=Qwen3, Setting=SFT-ind.2026.05 | 45 | 1.478 | 0.307 | 0.891 | 0.582 | 0.13 | 93.6 | 92.9 | 0.936 | 51.2 | — | |
| SFT-ind.Model=Llama3.2, Setting=SFT-ind.2026.05 | 45 | 1.535 | 0.29 | 0.889 | 0.57 | 0.118 | 96.9 | 96.3 | 0.972 | 51.2 | — | |
| SFTBackbone=Qwen32026.05 | 45 | 1.478 | — | — | — | 0.13 | — | 92.9 | 0.936 | 51.2 | 0.307 | |
| SFTBackbone=Llama3.22026.05 | 45 | 1.535 | — | — | — | 0.118 | — | 96.3 | 0.972 | 51.2 | 0.29 | |
| SFT-sharedModel=Qwen3, Setting=SFT-shared2026.05 | 44.3 | 1.617 | 0.304 | 0.89 | 0.569 | 0.22 | 80.7 | 80 | 0.806 | 45.5 | — | |
| VP-ICLModel=Qwen3, Setting=VP-ICL2026.05 | 41.5 | 1.667 | 0.276 | 0.885 | 0.601 | 0.246 | 93.8 | 94.7 | 0.956 | 51 | — | |
| VP-ICLBackbone=Qwen32026.05 | 41.5 | 1.667 | — | — | — | 0.246 | — | 94.7 | 0.956 | 51 | 0.276 | |
| ICLModel=Qwen3, Setting=ICL2026.05 | 40.7 | 1.702 | 0.197 | 0.871 | 0.535 | 0.587 | 80 | 80.9 | 0.81 | 46.3 | — | |
| ICLBackbone=Qwen32026.05 | 40.7 | 1.702 | — | — | — | 0.587 | — | 80.9 | 0.81 | 46.3 | 0.197 | |
| SFT-sharedModel=Llama3.2, Setting=SFT-shared2026.05 | 38.5 | 2.05 | 0.209 | 0.87 | 0.425 | 0.683 | 30.5 | 30.5 | 0.191 | 26.5 | — | |
| BaseModel=Qwen3, Setting=Base2026.05 | 29.2 | 2.453 | 0.071 | 0.84 | 0.395 | 5.188 | 24.8 | 24.8 | -0.004 | 23.7 | — | |
| BaseBackbone=Qwen32026.05 | 29.2 | 2.453 | — | — | — | 5.188 | — | 24.8 | -0.004 | 23.7 | 0.071 | |
| VPModel=Qwen3, Setting=VP2026.05 | 27.8 | 2.475 | 0.1 | 0.85 | 0.48 | 2.74 | 65.7 | 64.9 | 0.651 | 36 | — | |
| VPBackbone=Qwen32026.05 | 27.8 | 2.475 | — | — | — | 2.74 | — | 64.9 | 0.651 | 36 | 0.1 | |
| ICLModel=Llama3.2, Setting=ICL2026.05 | 27.3 | 3.333 | 0.213 | 0.875 | 0.546 | 0.48 | 87.5 | 87.1 | 0.879 | 36.7 | — | |
| VP-ICLModel=Llama3.2, Setting=VP-ICL2026.05 | 27.3 | 3.39 | 0.23 | 0.88 | 0.558 | 0.257 | 87.5 | 85.8 | 0.865 | 37 | — | |
| ICLBackbone=Llama3.22026.05 | 27.3 | 3.333 | — | — | — | 0.48 | — | 87.1 | 0.879 | 36.7 | 0.213 | |
| VP-ICLBackbone=Llama3.22026.05 | 27.3 | 3.39 | — | — | — | 0.257 | — | 85.8 | 0.865 | 37 | 0.23 | |
| BaseModel=Llama3.2, Setting=Base2026.05 | 20.7 | 3.188 | 0.068 | 0.835 | 0.38 | 1.515 | 25.1 | 25.5 | 0.015 | 23 | — | |
| BaseBackbone=Llama3.22026.05 | 20.7 | 3.188 | — | — | — | 1.515 | — | 25.5 | 0.015 | 23 | 0.068 | |
| VPModel=Llama3.2, Setting=VP2026.05 | 18 | 4.185 | 0.098 | 0.85 | 0.401 | 0.913 | 34.7 | 35.9 | 0.175 | 26 | — | |
| VPBackbone=Llama3.22026.05 | 18 | 4.185 | — | — | — | 0.913 | — | 35.9 | 0.175 | 26 | 0.098 |