Counterexample Generation on SAIR hard3 Official (n=20)
65.3AccuracyAN38
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| AN38Variant=AN38, Backbone=GPT-OSS 120B, Sample size (n)=20, Evaluation Pipeline=OpenRouter/DeepInfra, Precision=bf16, Temperature=0.0, Seed=0, Max tokens=8,1922026.04 | 65.3 | 5.6 | 67.6 | |
| BaselineVariant=no cheatsheet, Backbone=GPT-OSS 120B, Sample size (n)=20, Evaluation Pipeline=OpenRouter/DeepInfra, Precision=bf16, Temperature=0.0, Seed=0, Max tokens=8,1922026.04 | 56.3 | — | 53 | |
| AN45cVariant=AN45c, Backbone=GPT-OSS 120B, Sample size (n)=20, Evaluation Pipeline=OpenRouter/DeepInfra, Precision=bf16, Temperature=0.0, Seed=0, Max tokens=8,1922026.04 | 55.5 | 4.3 | 63.7 |