Science Question Answering on GPQA
87.6pass@1 AccuracyM2CL
Evaluation results
| Method | Links | |
|---|---|---|
| M2CLBackbone=Llama-70B, Number of LLMs=82026.02 | 87.6 | |
| M2CLBackbone=Qwen-72B, Number of LLMs=82026.02 | 78.9 | |
| UMEM-Qwen3-4BFrozen Executor=Gemini-2.5-Flash, Parameters=4B2026.02 | 76.26 | |
| MempFrozen Executor=Gemini-2.5-Flash, Parameters=API2026.02 | 74.75 | |
| No MemoryFrozen Executor=Gemini-2.5-Flash, Parameters=-2026.02 | 73.23 | |
| Llama-3.2-1B-InstructFrozen Executor=Gemini-2.5-Flash, Parameters=1B2026.02 | 73.23 | |
| DyLANBackbone=Llama-70B, Number of LLMs=82026.02 | 72.8 | |
| Qwen3-4B-InstructFrozen Executor=Gemini-2.5-Flash, Parameters=4B2026.02 | 72.22 | |
| Self-RAGFrozen Executor=Gemini-2.5-Flash, Parameters=API2026.02 | 71.72 | |
| UMEM-Llama-3.2-1BFrozen Executor=Gemini-2.5-Flash, Parameters=1B2026.02 | 71.72 | |
| GPTSwarmBackbone=Llama-70B, Number of LLMs=82026.02 | 71.2 | |
| M2CLBackbone=Llama-14B, Number of LLMs=82026.02 | 70.7 | |
| ReMemFrozen Executor=Gemini-2.5-Flash, Parameters=API2026.02 | 70.2 | |
| M2CLBackbone=Qwen-14B, Number of LLMs=82026.02 | 66.2 | |
| M2CLBackbone=Qwen-7B, Number of LLMs=82026.02 | 66.1 | |
| CoMBase Model=Gemini-2.0-Flash2026.02 | 65.7 | |
| MRPBase Model=Gemini-2.0-Flash2026.02 | 65.15 | |
| UMEM-Qwen3-4BFrozen Executor=GPT-5.1, Parameters=4B2026.02 | 65.15 | |
| Zero-shot CoTBase Model=Gemini-2.0-Flash2026.02 | 64.14 | |
| Direct I/OBase Model=Gemini-2.0-Flash2026.02 | 62.63 | |
| ReMemFrozen Executor=GPT-5.1, Parameters=API2026.02 | 62.63 | |
| UMEM-Llama-3.2-1BFrozen Executor=GPT-5.1, Parameters=1B2026.02 | 62.63 | |
| Qwen3-4B-InstructFrozen Executor=GPT-5.1, Parameters=4B2026.02 | 62.63 | |
| MacNetBackbone=Llama-70B, Number of LLMs=82026.02 | 62.2 | |
| MempFrozen Executor=GPT-5.1, Parameters=API2026.02 | 62.12 | |
| ReActBase Model=Gemini-2.0-Flash2026.02 | 61.62 | |
| Llama-3.2-1B-InstructFrozen Executor=GPT-5.1, Parameters=1B2026.02 | 61.11 | |
| BoNBackbone=Llama-70B, Number of LLMs=82026.02 | 59.6 | |
| M2CLBackbone=Llama-7B, Number of LLMs=82026.02 | 59.3 | |
| DebateBackbone=Llama-70B, Number of LLMs=82026.02 | 58.1 | |
| Self-RAGFrozen Executor=GPT-5.1, Parameters=API2026.02 | 57.58 | |
| No MemoryFrozen Executor=GPT-5.1, Parameters=-2026.02 | 57.57 | |
| UMEM-Llama-3.2-1BFrozen Executor=Qwen3-8B-Thinking, Parameters=1B2026.02 | 54.04 | |
| MacNetBackbone=Llama-14B, Number of LLMs=82026.02 | 53.9 | |
| Meta-ReasonerBase Model=Gemini-2.0-Flash2026.02 | 53.54 | |
| ReMemFrozen Executor=Qwen3-8B-Thinking, Parameters=8B2026.02 | 53.54 | |
| Qwen3-4B-InstructFrozen Executor=Qwen3-8B-Thinking, Parameters=4B2026.02 | 53.54 | |
| No MemoryFrozen Executor=Qwen3-8B-Thinking, Parameters=-2026.02 | 52.53 | |
| GPTSwarmBackbone=Qwen-72B, Number of LLMs=82026.02 | 52.4 | |
| Llama-3.2-1B-InstructFrozen Executor=Qwen3-8B-Thinking, Parameters=1B2026.02 | 52.02 | |
| UMEM-Qwen3-4BFrozen Executor=Qwen3-8B-Thinking, Parameters=4B2026.02 | 52.02 | |
| DyLANBackbone=Qwen-72B, Number of LLMs=82026.02 | 51.6 | |
| MempFrozen Executor=Qwen3-8B-Thinking, Parameters=8B2026.02 | 49.49 | |
| DyLANBackbone=Llama-14B, Number of LLMs=82026.02 | 46.7 | |
| Tree of ThoughtsBase Model=Gemini-2.0-Flash2026.02 | 46.46 | |
| MacNetBackbone=Qwen-72B, Number of LLMs=82026.02 | 46.2 | |
| BoNBackbone=Qwen-72B, Number of LLMs=82026.02 | 45.9 | |
| Self-RAGFrozen Executor=Qwen3-8B-Thinking, Parameters=8B2026.02 | 43.94 | |
| DebateBackbone=Qwen-72B, Number of LLMs=82026.02 | 43.4 | |
| MacNetBackbone=Llama-7B, Number of LLMs=82026.02 | 39.6 | |
| DyLANBackbone=Qwen-14B, Number of LLMs=82026.02 | 39.4 | |
| GPTSwarmBackbone=Qwen-14B, Number of LLMs=82026.02 | 38.7 | |
| Qwen 3 14BCompression Ratio=Full2025.09 | 38.6 | |
| BoNBackbone=Llama-14B, Number of LLMs=82026.02 | 37.7 | |
| Chain of CodeBase Model=Gemini-2.0-Flash2026.02 | 37.4 | |
| BoNBackbone=Qwen-7B, Number of LLMs=82026.02 | 36.4 | |
| Qwen 3 8BCompression Ratio=Full2025.09 | 36.2 | |
| GPTSwarmBackbone=Qwen-7B, Number of LLMs=82026.02 | 35.6 | |
| DyLANBackbone=Qwen-7B, Number of LLMs=82026.02 | 35.4 | |
| DyLANBackbone=Llama-7B, Number of LLMs=82026.02 | 35 | |
| SingleBackbone=Qwen-72B, Number of LLMs=82026.02 | 34.9 | |
| DebateBackbone=Llama-14B, Number of LLMs=82026.02 | 34.5 | |
| BoNBackbone=Qwen-14B, Number of LLMs=82026.02 | 32.8 | |
| MacNetBackbone=Qwen-14B, Number of LLMs=82026.02 | 31.9 | |
| MacNetBackbone=Qwen-7B, Number of LLMs=82026.02 | 30.8 | |
| CoSpaDiBase Model=Qwen 3 14B, Compression Ratio=0.22025.09 | 30.6 | |
| DebateBackbone=Qwen-14B, Number of LLMs=82026.02 | 30.3 | |
| SVDLLMBase Model=Qwen 3 14B, Compression Ratio=0.22025.09 | 29.2 | |
| GPTSwarmBackbone=Llama-7B, Number of LLMs=82026.02 | 28.7 | |
| DebateBackbone=Qwen-7B, Number of LLMs=82026.02 | 28.7 | |
| CoSpaDiBase Model=Qwen 3 8B, Compression Ratio=0.22025.09 | 28.6 | |
| SVDLLMBase Model=Qwen 3 8B, Compression Ratio=0.22025.09 | 28.4 | |
| SingleBackbone=Llama-70B, Number of LLMs=82026.02 | 27.6 | |
| BoNBackbone=Llama-7B, Number of LLMs=82026.02 | 27.5 | |
| SVDLLMBase Model=Qwen 3 14B, Compression Ratio=0.42025.09 | 26.4 | |
| CoSpaDiBase Model=Qwen 3 8B, Compression Ratio=0.42025.09 | 26 | |
| SVDLLMBase Model=Qwen 3 8B, Compression Ratio=0.32025.09 | 25.6 | |
| CoSpaDiBase Model=Qwen 3 14B, Compression Ratio=0.42025.09 | 25.5 | |
| SingleBackbone=Llama-14B, Number of LLMs=82026.02 | 25 | |
| DebateBackbone=Llama-7B, Number of LLMs=82026.02 | 24.8 | |
| CoSpaDiBase Model=Qwen 3 8B, Compression Ratio=0.32025.09 | 24.8 | |
| SVDLLMBase Model=Qwen 3 8B, Compression Ratio=0.42025.09 | 23.1 | |
| SingleBackbone=Qwen-14B, Number of LLMs=82026.02 | 21.2 | |
| SingleBackbone=Qwen-7B, Number of LLMs=82026.02 | 20.2 | |
| SingleBackbone=Llama-7B, Number of LLMs=82026.02 | 16.8 |