Mathematical Reasoning on GPQA Diamond (test)
67.34Accuracy@5LARK
Evaluation Results
| Method | Links | |
|---|---|---|
| LARKBackbone=Qwen-2.5-7B, Selection Budget (B)=32026.05 | 67.34 | |
| LARKBackbone=Qwen-2.5-1.5B, Selection Budget (B)=32026.05 | 66.67 | |
| LARKBackbone=Qwen-2.5-7B, Selection Budget (B)=12026.05 | 65.99 | |
| LARKBackbone=Qwen-2.5-1.5B, Selection Budget (B)=12026.05 | 63.13 | |
| RandomBackbone=Qwen-2.5-7B, Selection Budget (B)=32026.05 | 59.76 | |
| RandomBackbone=Qwen-2.5-7B, Selection Budget (B)=12026.05 | 56.9 | |
| RSRminBackbone=Qwen-2.5-7B, Selection Budget (B)=32026.05 | 56.73 | |
| Local NaturalnessmaxBackbone=Qwen-2.5-7B, Selection Budget (B)=32026.05 | 56.23 | |
| RandomBackbone=Qwen-2.5-1.5B, Selection Budget (B)=32026.05 | 55.89 | |
| GRAPEmaxBackbone=Qwen-2.5-7B, Selection Budget (B)=32026.05 | 55.05 | |
| LARKBackbone=Llama-3.2-3B, Selection Budget (B)=32026.05 | 54.55 | |
| RandomBackbone=Qwen-2.5-1.5B, Selection Budget (B)=12026.05 | 54.38 | |
| Rule-based QualitymaxBackbone=Qwen-2.5-7B, Selection Budget (B)=32026.05 | 53.54 | |
| LLM-judged QualitymaxBackbone=Qwen-2.5-7B, Selection Budget (B)=32026.05 | 53.03 | |
| RSRminBackbone=Qwen-2.5-7B, Selection Budget (B)=12026.05 | 52.19 | |
| Local NaturalnessmaxBackbone=Qwen-2.5-7B, Selection Budget (B)=12026.05 | 51.18 | |
| LLM-judged QualitymaxBackbone=Qwen-2.5-7B, Selection Budget (B)=12026.05 | 50.34 | |
| GRAPEmaxBackbone=Qwen-2.5-7B, Selection Budget (B)=12026.05 | 49.33 | |
| LARKBackbone=Llama-3.2-3B, Selection Budget (B)=12026.05 | 48.32 | |
| Rule-based QualitymaxBackbone=Qwen-2.5-7B, Selection Budget (B)=12026.05 | 48.15 | |
| RSRminBackbone=Qwen-2.5-1.5B, Selection Budget (B)=12026.05 | 47.14 | |
| Local NaturalnessmaxBackbone=Qwen-2.5-1.5B, Selection Budget (B)=32026.05 | 46.8 | |
| GRAPEmaxBackbone=Qwen-2.5-1.5B, Selection Budget (B)=32026.05 | 46.63 | |
| Rule-based QualitymaxBackbone=Qwen-2.5-1.5B, Selection Budget (B)=32026.05 | 46.3 | |
| LLM-judged QualitymaxBackbone=Qwen-2.5-1.5B, Selection Budget (B)=32026.05 | 46.3 | |
| RSRminBackbone=Qwen-2.5-1.5B, Selection Budget (B)=32026.05 | 46.3 | |
| Local NaturalnessmaxBackbone=Qwen-2.5-1.5B, Selection Budget (B)=12026.05 | 45.62 | |
| RandomBackbone=Llama-3.2-3B, Selection Budget (B)=32026.05 | 44.61 | |
| Token LengthmaxBackbone=Qwen-2.5-7B, Selection Budget (B)=32026.05 | 43.77 | |
| RSRminBackbone=Llama-3.2-3B, Selection Budget (B)=32026.05 | 43.77 | |
| LLM-judged QualitymaxBackbone=Llama-3.2-3B, Selection Budget (B)=32026.05 | 43.43 | |
| LLM-judged QualitymaxBackbone=Qwen-2.5-1.5B, Selection Budget (B)=12026.05 | 42.93 | |
| GRAPEmaxBackbone=Qwen-2.5-1.5B, Selection Budget (B)=12026.05 | 42.93 | |
| Local NaturalnessmaxBackbone=Llama-3.2-3B, Selection Budget (B)=32026.05 | 42.76 | |
| RSRminBackbone=Llama-3.2-3B, Selection Budget (B)=12026.05 | 42.09 | |
| Token LengthmaxBackbone=Qwen-2.5-1.5B, Selection Budget (B)=32026.05 | 42.09 | |
| LLM-judged QualitymaxBackbone=Llama-3.2-3B, Selection Budget (B)=12026.05 | 41.58 | |
| Rule-based QualitymaxBackbone=Qwen-2.5-1.5B, Selection Budget (B)=12026.05 | 41.25 | |
| Token LengthmaxBackbone=Qwen-2.5-7B, Selection Budget (B)=12026.05 | 41.08 | |
| Local NaturalnessmaxBackbone=Llama-3.2-3B, Selection Budget (B)=12026.05 | 41.08 | |
| RandomBackbone=Llama-3.2-3B, Selection Budget (B)=12026.05 | 40.4 | |
| BaseModelBackbone=Qwen-2.5-7B, Selection Budget (B)=12026.05 | 39.56 | |
| BaseModelBackbone=Qwen-2.5-7B, Selection Budget (B)=32026.05 | 39.56 | |
| Token LengthmaxBackbone=Llama-3.2-3B, Selection Budget (B)=32026.05 | 38.72 | |
| Token LengthmaxBackbone=Llama-3.2-3B, Selection Budget (B)=12026.05 | 38.22 | |
| Token LengthmaxBackbone=Qwen-2.5-1.5B, Selection Budget (B)=12026.05 | 34.51 | |
| BaseModelBackbone=Qwen-2.5-1.5B, Selection Budget (B)=12026.05 | 30.64 | |
| BaseModelBackbone=Qwen-2.5-1.5B, Selection Budget (B)=32026.05 | 30.64 | |
| Rule-based QualitymaxBackbone=Llama-3.2-3B, Selection Budget (B)=32026.05 | 23.57 | |
| GRAPEmaxBackbone=Llama-3.2-3B, Selection Budget (B)=32026.05 | 21.55 | |
| Rule-based QualitymaxBackbone=Llama-3.2-3B, Selection Budget (B)=12026.05 | 14.48 | |
| GRAPEmaxBackbone=Llama-3.2-3B, Selection Budget (B)=12026.05 | 14.31 | |
| BaseModelBackbone=Llama-3.2-3B, Selection Budget (B)=12026.05 | 11.45 | |
| BaseModelBackbone=Llama-3.2-3B, Selection Budget (B)=32026.05 | 11.45 |