Indirect Object Identification on IOI (CPR, CMD, Avg Faithfulness)
3.491CPREAP-IG
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| EAP-IGModel=Gemma-2-2B, Path=inputs2026.06 | 3.491 | 2.496 | 2.592 | |
| EAP-IGModel=Llama-3.1-8B, Path=inputs2026.06 | 2.391 | 1.392 | 2.404 | |
| EAP-IGModel=GPT-2, Path=inputs2026.06 | 2.155 | 1.159 | 1.796 | |
| HVPModel=Gemma-2-2B2026.06 | 1.91 | 0.927 | 1.125 | |
| EAP-IGModel=Qwen2.5-0.5B, Path=inputs2026.06 | 1.68 | 0.682 | 1.537 | |
| MS-HVPModel=Llama-3.1-8B, K=52026.06 | 1.376 | 0.379 | 1.129 | |
| EAPModel=Gemma-2-2B2026.06 | 1.364 | 0.38 | 0.9 | |
| MS-HVPModel=Gemma-2-2B, K=52026.06 | 1.306 | 0.327 | 0.814 | |
| HVPModel=GPT-22026.06 | 1.281 | 0.292 | 0.888 | |
| EAPModel=GPT-22026.06 | 1.267 | 0.278 | 0.885 | |
| MS-HVPModel=GPT-2, K=52026.06 | 1.253 | 0.265 | 0.871 | |
| EAPModel=Llama-3.1-8B2026.06 | 0.969 | 0.031 | 0.767 | |
| HVPModel=Llama-3.1-8B2026.06 | 0.939 | 0.06 | 0.75 | |
| HVPModel=Qwen2.5-0.5B2026.06 | 0.264 | 0.735 | 0.107 | |
| EAPModel=Qwen2.5-0.5B2026.06 | 0.263 | 0.736 | 0.107 | |
| MS-HVPModel=Qwen2.5-0.5B, K=52026.06 | 0.255 | 0.744 | 0.102 |