Cultural error evaluation on ExCAM40k (test)
59.1S.Acc.ExCAMr
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ExCAMrBackbone=Gemma3, Output style=report style2026.05 | 59.1 | 56.9 | 68 | |
| ExCAMrBackbone=Phi3, Output style=report style2026.05 | 49.2 | 49.3 | 61.7 | |
| Phi-4Prompting strategy=counting prompt2026.05 | 35.1 | 34.9 | 49.5 | |
| Phi-4Prompting strategy=counting prompt, Scoring method=sequence probability fallback (+p)2026.05 | 34.6 | 31.8 | 49.1 | |
| DS-R1-14BPrompting strategy=counting prompt2026.05 | 33.1 | 30.5 | 47.4 | |
| Gemma3-27BPrompting strategy=counting prompt2026.05 | 31.1 | 28.4 | 45.5 | |
| Mistral-24BPrompting strategy=counting prompt2026.05 | 28.2 | 24.6 | 45.2 | |
| Qwen3-14BPrompting strategy=counting prompt2026.05 | 28 | 28.5 | 46.4 | |
| Phi-3m-4kPrompting strategy=counting prompt2026.05 | 27 | 30.3 | 45.8 |