List your company
ResearchFormerly WizWand

Ternary Claim Verification on AVeriTeC (test)

48.5Balanced AccuracyQwen direct GRPO
40.1842.3444.546.66May 19, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Qwen direct GRPOtype=Non-Argumentative, variant=GRPO2026.0548.5
Qwen direct GRPOTraining Strategy=GRPO2026.0548.5
Qwen args + BSM-Q (primed & sem & rank & con)type=Base Score Model, base_score_model=BSM-Q, training_objectives=primed & sem & rank & con2026.0547.1
Qwen args + BSM-Q (primed & sem & rank & con)Argument Generation=Qwen3-8B, Base Score Model=BSM-Q, Loss Configuration=primed & sem & rank & con2026.0547.1
Qwen GRPO args + BSM-Q (primed & sem & rank & con)type=Combination of Models, argument_generation=GRPO-trained Qwen, base_score_model=BSM-Q, training_objectives=primed & sem & rank & con2026.0546.8
Qwen GRPO args + BSM-Q (primed & sem & rank & con)Argument Generation=GRPO, Base Score Model=BSM-Q, Loss Configuration=primed & sem & rank & con2026.0546.8
Qwen args + BSM-G (primed & sem & rank & con)type=Base Score Model, base_score_model=BSM-G, training_objectives=primed & sem & rank & con2026.0546.6
Qwen args + BSM-G (primed & sem & rank & con)Argument Generation=Qwen3-8B, Base Score Model=BSM-G, Loss Configuration=primed & sem & rank & con2026.0546.6
Qwen args + BSM-G (primed & sem)type=Base Score Model, base_score_model=BSM-G, training_objectives=primed & sem2026.0546
Qwen args + BSM-G (primed & sem)Argument Generation=Qwen3-8B, Base Score Model=BSM-G, Loss Configuration=primed & sem2026.0546
Qwen GRPO args + BSM-G (primed & sem)type=Combination of Models, argument_generation=GRPO-trained Qwen, base_score_model=BSM-G, training_objectives=primed & sem2026.0545.8
Qwen GRPO args + BSM-G (primed & sem)Argument Generation=GRPO, Base Score Model=BSM-G, Loss Configuration=primed & sem2026.0545.8
Qwen direct baselinetype=Non-Argumentative, variant=baseline2026.0545.4
Qwen direct baselineTraining Strategy=Direct2026.0545.4
Qwen direct SFTtype=Non-Argumentative, variant=SFT2026.0544.6
Qwen direct SFTTraining Strategy=SFT2026.0544.6
Qwen GRPO args + Qwen base scorestype=Argument Generator Model, argument_generation=GRPO-trained Qwen, base_score=prompted Qwen3-8B2026.0542.6
Qwen GRPO args + Qwen base scoresArgument Generation=GRPO, Base Score Model=Qwen base scores2026.0542.6
Qwen args + BSM-Q (primed & sem)type=Base Score Model, base_score_model=BSM-Q, training_objectives=primed & sem2026.0541.5
Qwen args + BSM-Q (primed & sem)Argument Generation=Qwen3-8B, Base Score Model=BSM-Q, Loss Configuration=primed & sem2026.0541.5
Qwen args + BSM-G (primed)Argument Generation=Qwen3-8B, Base Score Model=BSM-G, Loss Configuration=primed2026.0541
Qwen args + BSM-Q (primed)Argument Generation=Qwen3-8B, Base Score Model=BSM-Q, Loss Configuration=primed2026.0540.8
Qwen args + Qwen base scorestype=Argumentative Baseline, argument_generation=Qwen3-8B, base_score=prompted Qwen3-8B2026.0540.6
Qwen args + Qwen base scoresArgument Generation=Qwen3-8B, Base Score Model=Qwen base scores2026.0540.6
Qwen args + constant 0.5 base scorestype=Argumentative Baseline, argument_generation=Qwen3-8B, base_score=fixed 0.52026.0540.5
Qwen args + constant 0.5 base scoresArgument Generation=Qwen3-8B, Base Score Model=Constant 0.52026.0540.5