Ternary Claim Verification on AVeriTeC (test)
48.5Balanced AccuracyQwen direct GRPO
Evaluation Results
| Method | Links | |
|---|---|---|
| Qwen direct GRPOtype=Non-Argumentative, variant=GRPO2026.05 | 48.5 | |
| Qwen direct GRPOTraining Strategy=GRPO2026.05 | 48.5 | |
| Qwen args + BSM-Q (primed & sem & rank & con)type=Base Score Model, base_score_model=BSM-Q, training_objectives=primed & sem & rank & con2026.05 | 47.1 | |
| Qwen args + BSM-Q (primed & sem & rank & con)Argument Generation=Qwen3-8B, Base Score Model=BSM-Q, Loss Configuration=primed & sem & rank & con2026.05 | 47.1 | |
| Qwen GRPO args + BSM-Q (primed & sem & rank & con)type=Combination of Models, argument_generation=GRPO-trained Qwen, base_score_model=BSM-Q, training_objectives=primed & sem & rank & con2026.05 | 46.8 | |
| Qwen GRPO args + BSM-Q (primed & sem & rank & con)Argument Generation=GRPO, Base Score Model=BSM-Q, Loss Configuration=primed & sem & rank & con2026.05 | 46.8 | |
| Qwen args + BSM-G (primed & sem & rank & con)type=Base Score Model, base_score_model=BSM-G, training_objectives=primed & sem & rank & con2026.05 | 46.6 | |
| Qwen args + BSM-G (primed & sem & rank & con)Argument Generation=Qwen3-8B, Base Score Model=BSM-G, Loss Configuration=primed & sem & rank & con2026.05 | 46.6 | |
| Qwen args + BSM-G (primed & sem)type=Base Score Model, base_score_model=BSM-G, training_objectives=primed & sem2026.05 | 46 | |
| Qwen args + BSM-G (primed & sem)Argument Generation=Qwen3-8B, Base Score Model=BSM-G, Loss Configuration=primed & sem2026.05 | 46 | |
| Qwen GRPO args + BSM-G (primed & sem)type=Combination of Models, argument_generation=GRPO-trained Qwen, base_score_model=BSM-G, training_objectives=primed & sem2026.05 | 45.8 | |
| Qwen GRPO args + BSM-G (primed & sem)Argument Generation=GRPO, Base Score Model=BSM-G, Loss Configuration=primed & sem2026.05 | 45.8 | |
| Qwen direct baselinetype=Non-Argumentative, variant=baseline2026.05 | 45.4 | |
| Qwen direct baselineTraining Strategy=Direct2026.05 | 45.4 | |
| Qwen direct SFTtype=Non-Argumentative, variant=SFT2026.05 | 44.6 | |
| Qwen direct SFTTraining Strategy=SFT2026.05 | 44.6 | |
| Qwen GRPO args + Qwen base scorestype=Argument Generator Model, argument_generation=GRPO-trained Qwen, base_score=prompted Qwen3-8B2026.05 | 42.6 | |
| Qwen GRPO args + Qwen base scoresArgument Generation=GRPO, Base Score Model=Qwen base scores2026.05 | 42.6 | |
| Qwen args + BSM-Q (primed & sem)type=Base Score Model, base_score_model=BSM-Q, training_objectives=primed & sem2026.05 | 41.5 | |
| Qwen args + BSM-Q (primed & sem)Argument Generation=Qwen3-8B, Base Score Model=BSM-Q, Loss Configuration=primed & sem2026.05 | 41.5 | |
| Qwen args + BSM-G (primed)Argument Generation=Qwen3-8B, Base Score Model=BSM-G, Loss Configuration=primed2026.05 | 41 | |
| Qwen args + BSM-Q (primed)Argument Generation=Qwen3-8B, Base Score Model=BSM-Q, Loss Configuration=primed2026.05 | 40.8 | |
| Qwen args + Qwen base scorestype=Argumentative Baseline, argument_generation=Qwen3-8B, base_score=prompted Qwen3-8B2026.05 | 40.6 | |
| Qwen args + Qwen base scoresArgument Generation=Qwen3-8B, Base Score Model=Qwen base scores2026.05 | 40.6 | |
| Qwen args + constant 0.5 base scorestype=Argumentative Baseline, argument_generation=Qwen3-8B, base_score=fixed 0.52026.05 | 40.5 | |
| Qwen args + constant 0.5 base scoresArgument Generation=Qwen3-8B, Base Score Model=Constant 0.52026.05 | 40.5 |