Question Answering on SimpleQA
95.3AccuracyLiner Pro Reasoning
Evaluation results
| Method | Links | |
|---|---|---|
| Liner Pro ReasoningSearch=true, Model Source=Closed-source2026.02 | 95.3 | |
| Perplexity Deep ResearchSearch=true, Model Source=Closed-source2026.02 | 93.9 | |
| ROMASearch=true, Model Source=Open-source, Backbone=GLM-4.62026.02 | 93.9 | |
| Kimi-ResearcherSearch=true, Model Source=Open-source2026.02 | 93.6 | |
| GLM-4.6Search=true, Model Source=Open-source2026.02 | 91.9 | |
| Linkup Web SearchSearch=true, Model Source=Closed-source2026.02 | 90.1 | |
| ExaSearch=true, Model Source=Closed-source2026.02 | 90 | |
| Open Deep Search (DeepSeek-R1)Search=true, Model Source=Open-source2026.02 | 88.3 | |
| Perplexity Sonar Reasoning ProSearch=true, Model Source=Closed-source2026.02 | 85.8 | |
| Perplexity SonarSearch=true, Model Source=Closed-source2026.02 | 77.3 | |
| HEARTModel=Deepseek-Reasoner2025.09 | 75.29 | |
| HEARTModel=Gemini 2.5 Flash2025.09 | 73.99 | |
| HEARTModel=Gemini 2.5 Pro2025.09 | 73.56 | |
| CoTModel=Deepseek-Reasoner2025.09 | 71.28 | |
| Self ReflectionModel=Deepseek-Reasoner2025.09 | 71.2 | |
| WaitModel=Deepseek-Reasoner2025.09 | 71.19 | |
| Self ReflectionModel=Gemini 2.5 Flash2025.09 | 67.43 | |
| WaitModel=Gemini 2.5 Flash2025.09 | 63.65 | |
| Self ReflectionModel=Gemini 2.5 Pro2025.09 | 63.51 | |
| CoTModel=Gemini 2.5 Pro2025.09 | 62.54 | |
| GPT-4.5Search=false, Model Source=Closed-source2026.02 | 62.5 | |
| WaitModel=Gemini 2.5 Pro2025.09 | 61.63 | |
| CoTModel=Gemini 2.5 Flash2025.09 | 58.51 | |
| Gemini 2.5 ProSearch=false, Model Source=Closed-source2026.02 | 52.9 | |
| Claude-3.7-SonnetSearch=false, Model Source=Closed-source2026.02 | 50 | |
| MHLP2025.06 | 48.6 | |
| CoT2025.06 | 47.8 | |
| System-Message2025.06 | 47.3 | |
| OpenAI-o1-12172025.01 | 47 | |
| TextGrad2025.06 | 46.6 | |
| HEARTModel=Claude 4 Sonnet2025.09 | 44.99 | |
| Gemini 2.0 ProSearch=false, Model Source=Closed-source2026.02 | 44.3 | |
| Paraphrasing2025.06 | 43.7 | |
| Grok 3 BetaSearch=false, Model Source=Closed-source2026.02 | 43.6 | |
| o1Search=false, Model Source=Closed-source2026.02 | 42.6 | |
| o1-previewSearch=false, Model Source=Closed-source2026.02 | 42.4 | |
| Self ReflectionModel=Claude 4 Sonnet2025.09 | 41.26 | |
| GPT-4o-2024-08-06Search=false, Model Source=Closed-source2026.02 | 40.1 | |
| WaitModel=Claude 4 Sonnet2025.09 | 39.53 | |
| GPT-4o-2024-05-13Search=false, Model Source=Closed-source2026.02 | 39 | |
| GPT-4o-2024-11-20Search=false, Model Source=Closed-source2026.02 | 38.8 | |
| CoTModel=Claude 4 Sonnet2025.09 | 38.26 | |
| GPT-4o-05132025.01 | 38.2 | |
| GPT-4oSearch=false, Model Source=Closed-source2026.02 | 38.2 | |
| HEARTModel=GPT-5 nano2025.09 | 36.99 | |
| WaitModel=GPT-5 nano2025.09 | 36.45 | |
| CoTModel=GPT-5 nano2025.09 | 36.01 | |
| Kimi-K2 Base# Shots=5-shot, # Activated Params=32B, # Total Params=1043B2026.01 | 35.3 | |
| VanillaModel=Gemini 2.5 Pro2025.09 | 34.27 | |
| VanillaModel=Gemini 2.5 Flash2025.09 | 33.83 | |
| VanillaModel=Deepseek-Reasoner2025.09 | 32.25 | |
| Self ReflectionModel=GPT-5 nano2025.09 | 31.54 | |
| DeepSeek-R1Architecture=MoE, Activated Params=37B, Total Params=671B2025.01 | 30.1 | |
| DeepSeek-R1Search=false, Model Source=Open-source2026.02 | 30.1 | |
| Claude 3.5 SonnetSearch=false, Model Source=Closed-source2026.02 | 28.9 | |
| Claude-3.5-Sonnet-10222025.01 | 28.4 | |
| DeepSeek-R1-0528Quantization=FP8-Block-Wise2026.02 | 27.8 | |
| DeepSeek-V3.2 Exp Base# Shots=5-shot, # Activated Params=37B, # Total Params=671B2026.01 | 27 | |
| DeepSeek-R1-0528Quantization=W4A8-FP82026.02 | 26.83 | |
| DeepSeek-V3.1 Base# Shots=5-shot, # Activated Params=37B, # Total Params=671B2026.01 | 26.3 | |
| DeepSeek-V3Architecture=MoE, Activated Params=37B, Total Params=671B2025.01 | 24.9 | |
| DeepSeek-V3Search=false, Model Source=Open-source2026.02 | 24.9 | |
| GPT-4-turbo-2024-04-09Search=false, Model Source=Closed-source2026.02 | 24.2 | |
| VanillaModel=Claude 4 Sonnet2025.09 | 24.13 | |
| Claude 3 OpusSearch=false, Model Source=Closed-source2026.02 | 23.5 | |
| Grok 3 mini BetaSearch=false, Model Source=Closed-source2026.02 | 21.7 | |
| MiMo-V2-Flash Base# Shots=5-shot, # Activated Params=15B, # Total Params=309B2026.01 | 20.6 | |
| Llama-3.1-70BSearch=false, Model Source=Open-source2026.02 | 20.4 | |
| HEARTModel=Gemma3 12b Instruct2025.09 | 19.94 | |
| WaitModel=Gemma3 12b Instruct2025.09 | 17.71 | |
| HEARTModel=Gemma3 4b Instruct2025.09 | 15.61 | |
| o3-mini-highSearch=false, Model Source=Closed-source2026.02 | 13.8 | |
| o3-miniSearch=false, Model Source=Closed-source2026.02 | 13.4 | |
| o3-mini-lowSearch=false, Model Source=Closed-source2026.02 | 13 | |
| HaluSearch (MCTS)Backbone=Qwen2-7B-Instruct2025.01 | 12.5 | |
| CoTModel=Gemma3 12b Instruct2025.09 | 10.98 | |
| Self ReflectionModel=Gemma3 4b Instruct2025.09 | 10.82 | |
| VanillaModel=GPT-5 nano2025.09 | 10.81 | |
| GPT-4o-mini-2024-07-18Search=false, Model Source=Closed-source2026.02 | 9.5 | |
| WaitModel=Gemma3 4b Instruct2025.09 | 9.04 | |
| Self ReflectionModel=Gemma3 12b Instruct2025.09 | 8.61 | |
| HaluSearch (MCTS)Backbone=Llama3.1-8B-Instruct2025.01 | 8.5 | |
| CoTModel=Gemma3 4b Instruct2025.09 | 7.89 | |
| o1-miniSearch=false, Model Source=Closed-source2026.02 | 7.6 | |
| OpenAI-o1-mini2025.01 | 7 | |
| SCBackbone=Llama3.1-8B-Instruct2025.01 | 6 | |
| BoNBackbone=Qwen2-7B-Instruct2025.01 | 6 | |
| Self-RefineBackbone=Qwen2-7B-Instruct2025.01 | 6 | |
| RFTBackbone=Llama-3.1-8B2026.05 | 5.52 | |
| BoNBackbone=Llama3.1-8B-Instruct2025.01 | 5.5 | |
| Direct GenerationBackbone=Qwen2-7B-Instruct2025.01 | 5.5 | |
| BaseBackbone=Llama-3.1-8B2026.05 | 5.43 | |
| SFTBackbone=Llama-3.1-8B2026.05 | 5.43 | |
| DPOBackbone=Llama-3.1-8B2026.05 | 5.11 | |
| Self-RefineBackbone=Llama3.1-8B-Instruct2025.01 | 5 | |
| RLBackbone=Llama-3.1-8B2026.05 | 4.74 | |
| CoTBackbone=Qwen2-7B-Instruct2025.01 | 4.5 | |
| VanillaModel=Gemma3 12b Instruct2025.09 | 4.49 | |
| RLBackbone=OLMo-2-7B2026.05 | 4.33 | |
| BaseBackbone=Qwen2.5-7B2026.05 | 4.15 |