Expert-level Question Answering on GPQA
61.7Best AccuracyCISC
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| CISCLLM Backbone=Llama3.3 70B2026.05 | 61.7 | — | — | — | — | — | 61.7 | — | |
| VecCISC + KMeansLLM Backbone=Llama3.3 70B2026.05 | 61.7 | — | — | — | — | — | 61.6 | — | |
| VecCISC + HACLLM Backbone=Llama3.3 70B2026.05 | 61.7 | — | — | — | — | — | 61.7 | — | |
| SC BaselineLLM Backbone=Llama3.3 70B2026.05 | 60.6 | — | — | — | — | — | 60.6 | — | |
| VecCISC (random)LLM Backbone=Llama3.3 70B2026.05 | 58.6 | — | — | — | — | — | 57.3 | — | |
| VecCISC + KMeansLLM Backbone=GPT 4o-mini2026.05 | 41.4 | — | — | — | — | — | 39.9 | — | |
| VecCISC (random)LLM Backbone=GPT 4o-mini2026.05 | 40.7 | — | — | — | — | — | 39.2 | — | |
| CISCLLM Backbone=GPT 4o-mini2026.05 | 40.5 | — | — | — | — | — | 40.5 | — | |
| VecCISC + HACLLM Backbone=GPT 4o-mini2026.05 | 40.5 | — | — | — | — | — | 40.5 | — | |
| SC BaselineLLM Backbone=GPT 4o-mini2026.05 | 39.9 | — | — | — | — | — | 39.9 | — | |
| CISCLLM Backbone=Qwen2.5 7B2026.05 | 38.1 | — | — | — | — | — | 38.1 | — | |
| VecCISC (random)LLM Backbone=Qwen2.5 7B2026.05 | 38.1 | — | — | — | — | — | 35.4 | — | |
| VecCISC + KMeansLLM Backbone=Qwen2.5 7B2026.05 | 38.1 | — | — | — | — | — | 38.1 | — | |
| VecCISC + HACLLM Backbone=Qwen2.5 7B2026.05 | 38.1 | — | — | — | — | — | 38.1 | — | |
| SC BaselineLLM Backbone=Qwen2.5 7B2026.05 | 37.4 | — | — | — | — | — | 37.4 | — | |
| VecCISC + KMeansLLM Backbone=Llama3.1 8B2026.05 | 35.7 | — | — | — | — | — | 34.5 | — | |
| VecCISC + HACLLM Backbone=Llama3.1 8B2026.05 | 35.7 | — | — | — | — | — | 35.7 | — | |
| CISCLLM Backbone=Llama3.1 8B2026.05 | 35 | — | — | — | — | — | 35 | — | |
| SC BaselineLLM Backbone=Llama3.1 8B2026.05 | 34.8 | — | — | — | — | — | 34.8 | — | |
| VecCISC (random)LLM Backbone=Llama3.1 8B2026.05 | 33.9 | — | — | — | — | — | 30.5 | — | |
| VecCISC (random)LLM Backbone=Mistral 7B2026.05 | 28.8 | — | — | — | — | — | 27.1 | — | |
| CISCLLM Backbone=Mistral 7B2026.05 | 28.4 | — | — | — | — | — | 28.4 | — | |
| VecCISC + KMeansLLM Backbone=Mistral 7B2026.05 | 28.4 | — | — | — | — | — | 25.7 | — | |
| VecCISC + HACLLM Backbone=Mistral 7B2026.05 | 28.4 | — | — | — | — | — | 28.4 | — | |
| SC BaselineLLM Backbone=Mistral 7B2026.05 | 27.1 | — | — | — | — | — | 27.1 | — | |
| AAACModel=Qwen3.5-9B, Group Size (g)=162026.05 | — | — | — | — | — | — | — | 40 | |
| AAACModel=Qwen3.5-27B, Group Size (g)=162026.05 | — | — | — | — | — | — | — | 48.5 | |
| AWQModel=Qwen3.5-9B, Group Size (g)=162026.05 | — | — | — | — | — | — | — | 41.4 | |
| AWQModel=Qwen3.5-27B, Group Size (g)=162026.05 | — | — | — | — | — | — | — | 49.3 | |
| BF16Model=Qwen3.5-9B2026.05 | — | — | — | — | — | — | — | 42.2 | |
| BF16Model=Qwen3.5-27B2026.05 | — | — | — | — | — | — | — | 50.1 | |
| IF4Model=Qwen3.5-9B, Group Size (g)=162026.05 | — | — | — | — | — | — | — | 41.9 | |
| IF4Model=Qwen3.5-27B, Group Size (g)=162026.05 | — | — | — | — | — | — | — | 48.2 | |
| OptPO-SFTBackbone=Qwen2.5-Math-1.5B2025.12 | — | 27.6 | 78.8 | 31.3 | — | 51.17 | — | — | |
| OptPO-SFTBackbone=Llama-3.1-8B-Instruct2025.12 | — | 28.9 | 82.8 | 32.3 | — | 61.89 | — | — | |
| RTNModel=Qwen3.5-9B, Group Size (g)=162026.05 | — | — | — | — | — | — | — | 39.7 | |
| RTNModel=Qwen3.5-27B, Group Size (g)=162026.05 | — | — | — | — | — | — | — | 49.3 | |
| TTSFTBackbone=Qwen2.5-Math-1.5B2025.12 | — | 27.9 | 81.8 | 30.3 | — | — | — | — | |
| TTSFTBackbone=Llama-3.1-8B-Instruct2025.12 | — | 28.8 | 85.9 | 30.3 | — | — | — | — |