Ternary claim verification on TFU (test)
67.7AccuracyQwen args + BSM-G (primed & sem)
Evaluation Results
| Method | Links | |
|---|---|---|
| Qwen args + BSM-G (primed & sem)type=Base Score Model, base_score_model=BSM-G, training_objectives=primed & sem2026.05 | 67.7 | |
| Qwen args + BSM-G (primed & sem)Argument Generation=Qwen3-8B, Base Score Model=BSM-G, Loss Configuration=primed & sem2026.05 | 67.7 | |
| Qwen GRPO args + BSM-G (primed & sem)type=Combination of Models, argument_generation=GRPO-trained Qwen, base_score_model=BSM-G, training_objectives=primed & sem2026.05 | 66.8 | |
| Qwen GRPO args + BSM-G (primed & sem)Argument Generation=GRPO, Base Score Model=BSM-G, Loss Configuration=primed & sem2026.05 | 66.8 | |
| Qwen direct SFTtype=Non-Argumentative, variant=SFT2026.05 | 65.9 | |
| Qwen direct SFTTraining Strategy=SFT2026.05 | 65.9 | |
| Qwen direct baselinetype=Non-Argumentative, variant=baseline2026.05 | 63.5 | |
| Qwen direct baselineTraining Strategy=Direct2026.05 | 63.5 | |
| Qwen direct GRPOtype=Non-Argumentative, variant=GRPO2026.05 | 59.2 | |
| Qwen direct GRPOTraining Strategy=GRPO2026.05 | 59.2 | |
| Qwen GRPO args + Qwen base scorestype=Argument Generator Model, argument_generation=GRPO-trained Qwen, base_score=prompted Qwen3-8B2026.05 | 59.1 | |
| Qwen GRPO args + Qwen base scoresArgument Generation=GRPO, Base Score Model=Qwen base scores2026.05 | 59.1 | |
| Qwen args + BSM-Q (primed & sem & rank & con)type=Base Score Model, base_score_model=BSM-Q, training_objectives=primed & sem & rank & con2026.05 | 59 | |
| Qwen args + BSM-Q (primed & sem & rank & con)Argument Generation=Qwen3-8B, Base Score Model=BSM-Q, Loss Configuration=primed & sem & rank & con2026.05 | 59 | |
| Qwen args + BSM-Q (primed & sem)type=Base Score Model, base_score_model=BSM-Q, training_objectives=primed & sem2026.05 | 58.8 | |
| Qwen args + BSM-Q (primed & sem)Argument Generation=Qwen3-8B, Base Score Model=BSM-Q, Loss Configuration=primed & sem2026.05 | 58.8 | |
| Qwen args + BSM-G (primed)Argument Generation=Qwen3-8B, Base Score Model=BSM-G, Loss Configuration=primed2026.05 | 58.8 | |
| Qwen args + Qwen base scorestype=Argumentative Baseline, argument_generation=Qwen3-8B, base_score=prompted Qwen3-8B2026.05 | 58.5 | |
| Qwen args + Qwen base scoresArgument Generation=Qwen3-8B, Base Score Model=Qwen base scores2026.05 | 58.5 | |
| Qwen args + BSM-G (primed & sem & rank & con)type=Base Score Model, base_score_model=BSM-G, training_objectives=primed & sem & rank & con2026.05 | 58.4 | |
| Qwen args + BSM-G (primed & sem & rank & con)Argument Generation=Qwen3-8B, Base Score Model=BSM-G, Loss Configuration=primed & sem & rank & con2026.05 | 58.4 | |
| Qwen args + BSM-Q (primed)Argument Generation=Qwen3-8B, Base Score Model=BSM-Q, Loss Configuration=primed2026.05 | 58.3 | |
| Qwen GRPO args + BSM-Q (primed & sem & rank & con)type=Combination of Models, argument_generation=GRPO-trained Qwen, base_score_model=BSM-Q, training_objectives=primed & sem & rank & con2026.05 | 58.1 | |
| Qwen GRPO args + BSM-Q (primed & sem & rank & con)Argument Generation=GRPO, Base Score Model=BSM-Q, Loss Configuration=primed & sem & rank & con2026.05 | 58.1 | |
| Qwen args + constant 0.5 base scorestype=Argumentative Baseline, argument_generation=Qwen3-8B, base_score=fixed 0.52026.05 | 55.5 | |
| Qwen args + constant 0.5 base scoresArgument Generation=Qwen3-8B, Base Score Model=Constant 0.52026.05 | 55.5 |