List your company
ResearchFormerly WizWand

Scientific Reasoning on TheoremQA

82.49AccuracyUpper Bound
14.47432.13249.7967.448Aug 29, 2025Oct 4, 2025Nov 10, 2025Dec 17, 2025Jan 22, 2026Feb 28, 2026Apr 6, 2026
Updated 6mo ago

Evaluation results

MethodLinks
Upper BoundBackbone=Gemma-2-9B-Instruct, k=322025.0882.49—
Upper BoundBackbone=Gemma-2-9B-Instruct, k=62025.0855.22—
Upper BoundModel=DS-Distill-Qwen-2.5-7B, k=62025.0848.59—
Upper BoundModel=Qwen3-8B, k=62025.0844.71—
Upper BoundBackbone=Llama-3.1-70B-Instruct, k=162025.0843.47—
Upper BoundBackbone=Qwen3-32B (Non-thinking), k=162025.0842.93—
PiCSARModel=Qwen3-8B, k=62025.0842.57—
TABBudget (B)=10k2026.0442.36,566
StaticBudget=40962026.0442.110,754
Self-ConsistencyModel=Qwen3-8B, k=62025.0841.81—
TABBudget (B)=8k2026.0441.54,987
CISC (PiCSAR)Backbone=Llama-3.1-8B-Instruct, k=322025.0841.5—
TABBudget (B)=5k2026.0440.93,141
AverageModel=Qwen3-8B, k=62025.0840.83—
LLM-Judge Multi-TurnSelection Strategy=Multi-Turn2026.0440.85,375
Upper BoundBackbone=Llama-3.1-70B-Instruct, k=62025.0840.7—
StaticBudget=20482026.0440.35,988
Upper BoundBackbone=Qwen3-8B (Non-thinking), k=322025.0840.03—
StaticBudget=10242026.04403,975
TABBudget (B)=3k2026.0439.92,320
LLM-Judge IndividualSelection Strategy=Individual2026.0439.85,916
Upper BoundBackbone=Qwen3-32B (Non-thinking), k=62025.0839.76—
StaticBudget=5122026.0439.52,578
CISC (p(True))Backbone=Llama-3.1-8B-Instruct, k=322025.0839.38—
StaticBudget=2562026.0438.11,778
Upper BoundBackbone=Llama-3.1-8B-Instruct, k=322025.0837.846—
PiCSARModel=DS-Distill-Qwen-2.5-7B, k=62025.0836.76—
CICS (PiCSAR)Backbone=Qwen3-8B (Non-thinking), k=62025.0836.46—
CICS (PiCSAR)Backbone=Qwen3-32B (Non-thinking), k=62025.0836.46—
Upper BoundModel=DS-Distill-llama-3-8B, k=62025.0836.37—
CICS (PiCSAR)Backbone=Qwen3-8B (Non-thinking), k=322025.0836.32—
CICS (PiCSAR)Backbone=Qwen3-32B (Non-thinking), k=162025.0836.32—
Self-ConsistencyModel=DS-Distill-Qwen-2.5-7B, k=62025.0835.88—
Upper BoundBackbone=Qwen3-8B (Non-thinking), k=62025.0834.94—
AverageModel=DS-Distill-Qwen-2.5-7B, k=62025.0833.11—
CICS (p(True))Backbone=Qwen3-8B (Non-thinking), k=322025.0832.75—
CICS (P-True)Backbone=Qwen3-32B (Non-thinking), k=162025.0832.75—
CISC (PiCSAR)Backbone=Llama-3.1-70B-Instruct, k=62025.0831.91—
CISC (PiCSAR)Backbone=Llama-3.1-70B-Instruct, k=162025.0831.59—
PiCSARBackbone=Qwen3-32B (Non-thinking), k=62025.0831.46—
PiCSARModel=Qwen3-32B (Non-thinking), k=62025.0831.46—
PiCSARBackbone=Qwen3-32B (Non-thinking), k=162025.0831.42—
CICS (p(True))Backbone=Qwen3-8B (Non-thinking), k=62025.0831.24—
CICS (P-True)Backbone=Qwen3-32B (Non-thinking), k=62025.0831.24—
PiCSARBackbone=Qwen3-8B (Non-thinking), k=62025.0829.76—
PiCSARModel=Qwen3-8B (Non-thinking), k=62025.0829.76—
CISC (p(True))Backbone=Llama-3.1-70B-Instruct, k=162025.0829.41—
PiCSARBackbone=Qwen3-8B (Non-thinking), k=322025.0829.17—
Upper BoundBackbone=Llama-3.1-8B-Instruct, k=62025.0828.2—
CISC (p(True))Backbone=Llama-3.1-70B-Instruct, k=62025.0828.07—
PiCSARModel=DS-Distill-llama-3-8B, k=62025.0828.02—
Self-ConsistencyModel=DS-Distill-llama-3-8B, k=62025.0827.98—
PiCSARBackbone=Llama-3.1-70B-Instruct, k=62025.0827.84—
PiCSARModel=Llama-3.1-70B-Instruct, k=62025.0827.84—
PiCSARBackbone=Llama-3.1-70B-Instruct, k=162025.0826.73—
AverageModel=DS-Distill-llama-3-8B, k=62025.0826.58—
CISC (PiCSAR)Backbone=Gemma-2-9B-Instruct, k=322025.0823.16—
CISC (PiCSAR)Backbone=Llama-3.1-8B-Instruct, k=62025.0822.71—
CISC (PiCSAR)Backbone=Gemma-2-9B-Instruct, k=62025.0821.02—
PiCSARBackbone=Llama-3.1-8B-Instruct, k=62025.0820.08—
PiCSARModel=Llama-3.1-8B-Instruct, k=62025.0820.08—
PiCSARBackbone=Llama-3.1-8B-Instruct, k=322025.0819.72—
PiCSARBackbone=Gemma-2-9B-Instruct, k=322025.0818.88—
PiCSARBackbone=Gemma-2-9B-Instruct, k=62025.0818.62—
PiCSARModel=Gemma-2-9B-Instruct, k=62025.0818.62—
CISC (p(True))Backbone=Llama-3.1-8B-Instruct, k=62025.0818.03—
CISC (p(True))Backbone=Gemma-2-9B-Instruct, k=322025.0817.45—
CISC (p(True))Backbone=Gemma-2-9B-Instruct, k=62025.0817.09—