Factual Knowledge on BoolQ
87.7AccuracyORACLE
Evaluation results
| Method | Links | |
|---|---|---|
| ORACLEBackbone=Llama-3.1-8B-Instruct, Setting=zero-shot2026.03 | 87.7 | |
| ORACLEBackbone=Qwen-2.5-7B-instruct, Setting=zero-shot2026.03 | 87.5 | |
| ORACLEBackbone=Mistral-7B-Instruct-v0.3, Setting=zero-shot2026.03 | 86.8 | |
| RFTBackbone=Llama-3.1-8B-Instruct, Setting=zero-shot2026.03 | 86.3 | |
| self-rewardingBackbone=Llama-3.1-8B-Instruct, Setting=zero-shot2026.03 | 86.2 | |
| self-rewardingBackbone=Mistral-7B-Instruct-v0.3, Setting=zero-shot2026.03 | 85.6 | |
| CoTBackbone=Llama-3.1-8B-Instruct, Setting=zero-shot2026.03 | 85 | |
| RFTBackbone=Qwen-2.5-7B-instruct, Setting=zero-shot2026.03 | 84.7 | |
| RFTBackbone=Mistral-7B-Instruct-v0.3, Setting=zero-shot2026.03 | 84.4 | |
| self-rewardingBackbone=Qwen-2.5-7B-instruct, Setting=zero-shot2026.03 | 83.8 | |
| CoTBackbone=Llama-3.1-8B-Instruct, Setting=four-shot2026.03 | 83.2 | |
| CoTBackbone=Mistral-7B-Instruct-v0.3, Setting=zero-shot2026.03 | 83.1 | |
| CoTBackbone=Qwen-2.5-7B-instruct, Setting=four-shot2026.03 | 83 | |
| CoTBackbone=Qwen-2.5-7B-instruct, Setting=zero-shot2026.03 | 82.8 | |
| ToT-SFTBackbone=Llama-3.1-8B-Instruct, Setting=zero-shot2026.03 | 82.5 | |
| DenseLLM=Llama2-13B, Ratio=0.00%2024.03 | 82.39 | |
| ToT-SFTBackbone=Qwen-2.5-7B-instruct, Setting=zero-shot2026.03 | 82.1 | |
| Llama 3 8BParameters=8B, Shots=0-shot2024.10 | 80.9 | |
| ToT-SFTBackbone=Mistral-7B-Instruct-v0.3, Setting=zero-shot2026.03 | 80.9 | |
| CoTBackbone=Mistral-7B-Instruct-v0.3, Setting=four-shot2026.03 | 80.5 | |
| Phi-3 Mini-4kShots=0-shot2024.10 | 77.6 | |
| Mistral 8x7BParameters=8x7B, Shots=0-shot2024.10 | 76.6 | |
| ShortGPTLLM=Llama2-7B, Ratio=27.1%2024.03 | 74.71 | |
| DenseLLM=Baichuan2-7B, Ratio=0.00%2024.03 | 74.1 | |
| Mistral 7BParameters=7B, Shots=0-shot2024.10 | 72.2 | |
| DenseLLM=Llama2-7B, Ratio=0.00%2024.03 | 71.62 | |
| ShortGPTLLM=Baichuan2-7B, Ratio=24.2%2024.03 | 67.83 | |
| Gemma 7BParameters=7B, Shots=0-shot2024.10 | 66 | |
| DenseLLM=Baichuan2-13B, Ratio=0.00%2024.03 | 65.6 | |
| LaCoLLM=Llama2-7B, Ratio=27.1%2024.03 | 64.07 | |
| LaCoLLM=Llama2-13B, Ratio=24.6%2024.03 | 63.98 | |
| ShortGPTLLM=Baichuan2-13B, Ratio=24.7%2024.03 | 62.54 | |
| ShortGPTLLM=Llama2-13B, Ratio=24.6%2024.03 | 62.48 | |
| LaCoLLM=Baichuan2-13B, Ratio=24.7%2024.03 | 62.35 | |
| LLMPrun.LLM=Baichuan2-7B, Ratio=24.2%2024.03 | 61.19 | |
| Shakti-LLMParameters=2.5B, Shots=0-shot2024.10 | 61.1 | |
| LLMPrun.LLM=Baichuan2-13B, Ratio=24.3%2024.03 | 56.54 | |
| LLMPrun.LLM=Llama2-13B, Ratio=24.4%2024.03 | 56.42 | |
| LaCoLLM=Baichuan2-7B, Ratio=24.2%2024.03 | 56.15 | |
| LLMPrun.LLM=Llama2-7B, Ratio=27.0%2024.03 | 55.2 | |
| SliceGPTLLM=Baichuan2-7B, Ratio=22.2%2024.03 | 39.3 | |
| SliceGPTLLM=Llama2-7B, Ratio=26.4%2024.03 | 38.32 | |
| SliceGPTLLM=Llama2-13B, Ratio=23.6%2024.03 | 37.86 | |
| SliceGPTLLM=Baichuan2-13B, Ratio=22.8%2024.03 | 37.83 |