Graduate-level Question Answering on GPQA
96.9AccuracyM2CL
Evaluation results
| Method | Links | |
|---|---|---|
| M2CLModel=Llama-70B, Number of LLMs=642026.02 | 96.9 | |
| M2CLModel=Llama-14B, Number of LLMs=642026.02 | 93.1 | |
| M2CLModel=Qwen-72B, Number of LLMs=642026.02 | 86 | |
| MacNetModel=Llama-70B, Number of LLMs=642026.02 | 84.2 | |
| M2CLModel=Llama-7B, Number of LLMs=642026.02 | 82.5 | |
| M2CLModel=Qwen-7B, Number of LLMs=642026.02 | 80.6 | |
| M2CLModel=Qwen-14B, Number of LLMs=642026.02 | 80.4 | |
| GPTSwarmModel=Llama-70B, Number of LLMs=642026.02 | 78.9 | |
| DyLANModel=Llama-70B, Number of LLMs=642026.02 | 75.4 | |
| TAMELLM Backbone=GPT-5.22026.02 | 70.2 | |
| DebateModel=Llama-70B, Number of LLMs=642026.02 | 69 | |
| MIPROv2Model=o3 Mini2025.11 | 68.4 | |
| MacNetModel=Llama-14B, Number of LLMs=642026.02 | 67.5 | |
| Zero-Shot PredictModel=o3 Mini2025.11 | 66.6 | |
| Zero-Shot CoTModel=o3 Mini2025.11 | 66.4 | |
| ReasoningbankLLM Backbone=GPT-5.22026.02 | 65.9 | |
| BFRSModel=o3 Mini2025.11 | 65.5 | |
| Reasoningbank+promptLLM Backbone=GPT-5.22026.02 | 64.9 | |
| ReasoningbankLLM Backbone=Qwen3-32B2026.02 | 64.4 | |
| BFRSModel=Claude 3.7 Sonnet2025.11 | 64.1 | |
| Reasoningbank+promptLLM Backbone=Qwen3-32B2026.02 | 63.9 | |
| MacNetModel=Qwen-72B, Number of LLMs=642026.02 | 63.2 | |
| TAMELLM Backbone=Qwen3-32B2026.02 | 63.1 | |
| No-MemoryLLM Backbone=GPT-5.22026.02 | 63.1 | |
| MementoLLM Backbone=GPT-5.22026.02 | 63.1 | |
| Reasoningbank+GuardLLM Backbone=GPT-5.22026.02 | 63.1 | |
| Zero-Shot PredictModel=Claude 3.7 Sonnet2025.11 | 62.1 | |
| MIPROv2Model=Claude 3.7 Sonnet2025.11 | 61.9 | |
| Zero-Shot CoTModel=Claude 3.7 Sonnet2025.11 | 61.4 | |
| BFRSModel=Gemini 2.0 Flash2025.11 | 61 | |
| BoNModel=Llama-70B, Number of LLMs=642026.02 | 59.6 | |
| Qwen3 ModelParameters=235B-25072025.09 | 59.6 | |
| Reasoningbank+GuardLLM Backbone=Qwen3-32B2026.02 | 59.3 | |
| Zero-Shot CoTModel=Gemini 2.0 Flash2025.11 | 59.2 | |
| MIPROv2Model=Gemini 2.0 Flash2025.11 | 59 | |
| HELM BaselineModel=o3 Mini2025.11 | 57.6 | |
| HELM BaselineModel=Claude 3.7 Sonnet2025.11 | 57 | |
| BFRSModel=Llama 3.3 70B2025.11 | 56.3 | |
| Zero-Shot PredictModel=Llama 3.3 70B2025.11 | 55.8 | |
| Zero-Shot CoTModel=Llama 3.3 70B2025.11 | 55.8 | |
| DyLANModel=Qwen-72B, Number of LLMs=642026.02 | 55.7 | |
| GPT-4o + QuaSARBackbone=GPT-4o, Prompting Strategy=QuaSAR2025.02 | 55.4 | |
| GPTSwarmModel=Qwen-72B, Number of LLMs=642026.02 | 55.2 | |
| Zero-Shot PredictModel=Gemini 2.0 Flash2025.11 | 54.5 | |
| MacNetModel=Llama-7B, Number of LLMs=642026.02 | 54.4 | |
| HELM BaselineModel=Gemini 2.0 Flash2025.11 | 53.4 | |
| MIPROv2Model=Llama 3.3 70B2025.11 | 52.7 | |
| Zero-Shot CoTModel=GPT 4o2025.11 | 52.5 | |
| DebateModel=Qwen-72B, Number of LLMs=642026.02 | 52.4 | |
| MIPROv2Model=Qwen3 4B2025.11 | 52.2 | |
| MacNetModel=Qwen-7B, Number of LLMs=642026.02 | 50.4 | |
| GPT-4o + CoTBackbone=GPT-4o, Prompting Strategy=CoT2025.02 | 50.2 | |
| Zero-Shot CoTModel=Qwen3 4B2025.11 | 50 | |
| BFRSModel=GPT 4o2025.11 | 49.3 | |
| Llama-3-70B + QuaSARBackbone=Llama-3-70B, Prompting Strategy=QuaSAR2025.02 | 49.2 | |
| DyLANModel=Llama-14B, Number of LLMs=642026.02 | 48.8 | |
| Qwen2-72B + QuaSARBackbone=Qwen2-72B, Prompting Strategy=QuaSAR2025.02 | 48.2 | |
| BFRSModel=Qwen3 4B2025.11 | 47.8 | |
| MIPROv2Model=GPT 4o2025.11 | 47.8 | |
| MacNetModel=Qwen-14B, Number of LLMs=642026.02 | 46.7 | |
| GPT-4oBackbone=GPT-4o, Prompting Strategy=Zero-shot2025.02 | 46.5 | |
| BoNModel=Qwen-72B, Number of LLMs=642026.02 | 45.9 | |
| HELM BaselineModel=GPT 4o2025.11 | 45.5 | |
| Qwen3 ModelParameters=32B2025.09 | 45.45 | |
| DebateModel=Llama-14B, Number of LLMs=642026.02 | 45 | |
| Qwen3-8BModel Type=Causal LM, Few-shot examples=52026.05 | 44.4 | |
| DCLLM Backbone=GPT-5.22026.02 | 44 | |
| DyLANModel=Qwen-7B, Number of LLMs=642026.02 | 43.5 | |
| GPTSwarmModel=Qwen-14B, Number of LLMs=642026.02 | 43.2 | |
| GPTSwarmModel=Qwen-7B, Number of LLMs=642026.02 | 43.1 | |
| DyLANModel=Qwen-14B, Number of LLMs=642026.02 | 42.8 | |
| Qwen2-72BBackbone=Qwen2-72B, Prompting Strategy=Zero-shot2025.02 | 42.4 | |
| No-MemoryLLM Backbone=Qwen3-32B2026.02 | 42.2 | |
| Llama-3-70B + CoTBackbone=Llama-3-70B, Prompting Strategy=CoT2025.02 | 41.9 | |
| DebateModel=Qwen-7B, Number of LLMs=642026.02 | 41.7 | |
| Zero-Shot PredictModel=GPT 4o2025.11 | 41.7 | |
| Llama-3-70BBackbone=Llama-3-70B, Prompting Strategy=Zero-shot2025.02 | 41.3 | |
| MementoLLM Backbone=Qwen3-32B2026.02 | 40.5 | |
| HELM BaselineModel=Llama 3.3 70B2025.11 | 40.4 | |
| Self-ConsistencyModels=Qwen2.5-7B-Instruct, Qwen2.5-Coder-7B-Instruct, Mathstral-7B-v0.12026.05 | 40.4 | |
| DebateModel=Qwen-14B, Number of LLMs=642026.02 | 39.9 | |
| DarkForestModels=Qwen2.5-7B-Instruct, Qwen2.5-Coder-7B-Instruct, Mathstral-7B-v0.12026.05 | 39.9 | |
| Qwen3 ModelParameters=8B2025.09 | 39.9 | |
| Qwen2-72B + CoTBackbone=Qwen2-72B, Prompting Strategy=CoT2025.02 | 39.8 | |
| QuaSARBackbone=Llama-3-8B, Protocol=Annotation Tuning2025.02 | 39.2 | |
| FineRMoEBase Model=Qwen2.5-7B, #P/B=26.65, #AP/B=7.942026.03 | 39.06 | |
| BoNModel=Llama-14B, Number of LLMs=642026.02 | 37.7 | |
| GRPOBackbone=LLaMA-3.1-8B2025.05 | 37 | |
| DyLANModel=Llama-7B, Number of LLMs=642026.02 | 36.9 | |
| MDMType=Part Debate2025.07 | 36.6 | |
| Dream-7BModel Type=Diffusion LM, Few-shot examples=52026.05 | 36.6 | |
| BoNModel=Qwen-7B, Number of LLMs=642026.02 | 36.4 | |
| Qwen2.5-7BArchitecture=AR Models2026.04 | 36.4 | |
| LLaDA + Geometric AlignmentModel Type=dLLM, Gen Length=1282026.04 | 36.4 | |
| C32A2Base Model=Qwen2.5-7B, #P/B=184.42, #AP/B=13.332026.03 | 36.28 | |
| LLaDA + Outcome RLModel Type=dLLM, Gen Length=1282026.04 | 36.2 | |
| TPOBackbone=LLaMA-3.1-8B2025.05 | 36 | |
| DebateModels=Qwen2.5-7B-Instruct, Qwen2.5-Coder-7B-Instruct, Mathstral-7B-v0.12026.05 | 35.86 | |
| Mixture-of-AgentModels=Qwen2.5-7B-Instruct, Qwen2.5-Coder-7B-Instruct, Mathstral-7B-v0.12026.05 | 35.86 | |
| LLaMA3-8BModel Type=Causal LM, Few-shot examples=52026.05 | 35.5 |