Counterexample Generation on SAIR hard3 Local (n=400)
79.25AccuracyAN45c
Evaluation Results
| Method | Links | |
|---|---|---|
| AN45cVariant=AN45c, Backbone=GPT-OSS 120B, Sample size (n)=400, Evaluation Pipeline=Together AI, Precision=bf16, Temperature=0.0, Seed=42, Max tokens=8,1922026.04 | 79.25 | |
| AN38Variant=AN38, Backbone=GPT-OSS 120B, Sample size (n)=400, Evaluation Pipeline=Together AI, Precision=bf16, Temperature=0.0, Seed=42, Max tokens=8,1922026.04 | 71.8 | |
| BaselineVariant=no cheatsheet, Backbone=GPT-OSS 120B, Sample size (n)=400, Evaluation Pipeline=Together AI, Precision=bf16, Temperature=0.0, Seed=42, Max tokens=8,1922026.04 | 59.75 |