List your company

Variation error identification and evaluation on VariErr (Expanded)

63.7AccuracyQwen3
30.21238.90647.656.294May 27, 2026

Evaluation results

MethodLinks
Qwen3Setting=SFT-ind.2026.0563.728.389.2590.08480.584.50.85930
Qwen3Setting=SFT-shared2026.0563.224.488.756.20.2829.631.20.30826.3
Qwen3Setting=CAPO-strict2026.0562.728.189.158.20.08182.386.70.88832.8
Llama3.2Setting=SFT-shared2026.0557.522.988.355.40.37225.525.60.04825.5
Qwen3Setting=VP-ICL2026.0554.520.187.758.20.4765249.40.52828.2
Qwen3Setting=ICL2026.0553.518.687.355.90.69233.534.40.17926.5
Llama3.2Setting=SFT-ind.2026.0551.227.188.754.90.11387.390.90.94628.2
Llama3.2Setting=CAPO-strict2026.0551.226.288.7540.12188.592.40.96429.7
Qwen3Setting=VP2026.0549.213.185.751.74.39121.723.4-0.34824
Qwen3Setting=Base2026.0547.215.986.651.62.3942423.9-0.03423.7
Llama3.2Setting=VP-ICL2026.0541.22187.755.20.53926260.43227.5
Llama3.2Setting=ICL2026.053821.487.6550.47629.129.50.37526
Llama3.2Setting=VP2026.0533.814.886.346.71.90737.935.80.32525.3
Llama3.2Setting=Base2026.0531.515.386.548.32.03626.525.80.02326.5