Physics problem solving on Scibench fund
74.6AccuracyPoT + REFTOOL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| PoT + REFTOOLBackbone=GPT-4o2025.05 | 74.6 | — | |
| PoTBackbone=GPT-4o2025.05 | 73.2 | — | |
| PoT + RAGBackbone=Gemini-1.5-Pro2025.05 | 73.2 | — | |
| PoT + RAGBackbone=GPT-4o2025.05 | 73.2 | — | |
| PoT + REFTOOLBackbone=Gemini-1.5-Pro2025.05 | 73.2 | — | |
| Physics ReasonerBackbone=Gemini-1.5-Pro, Few-shot=4-shot2025.05 | 73.2 | — | |
| Physics ReasonerBackbone=GPT-4o, Few-shot=4-shot2025.05 | 71.8 | — | |
| CreatorBackbone=Gemini-1.5-Pro2025.05 | 70.4 | — | |
| PoTBackbone=Gemini-1.5-Pro2025.05 | 69 | — | |
| PoT + REFTOOLBackbone=GPT-42025.05 | 64.8 | — | |
| Physics ReasonerBackbone=GPT-42025.05 | 63.4 | — | |
| Physics ReasonerBackbone=GPT-4, Few-shot=4-shot2025.05 | 63.4 | — | |
| Physics ReasonerBackbone=Llama-3.1-70B, Few-shot=4-shot2025.05 | 62 | — | |
| Physics ReasonerBackbone=Gemini-1.5-Pro2025.05 | 59.2 | — | |
| PoTBackbone=GPT-42025.05 | 59.2 | — | |
| PoT + RAGBackbone=GPT-42025.05 | 59.2 | — | |
| CreatorBackbone=GPT-4o2025.05 | 57.7 | — | |
| PoT + REFTOOLBackbone=Llama-3.1-70B2025.05 | 57.7 | — | |
| Physics ReasonerBackbone=Llama-3.1-70B2025.05 | 56.3 | — | |
| CreatorBackbone=Llama-3.1-70B2025.05 | 56.3 | — | |
| PoT + RAGBackbone=Llama-3.1-70B2025.05 | 54.9 | — | |
| CreatorBackbone=GPT-42025.05 | 53.5 | — | |
| PoTBackbone=Llama-3.1-70B2025.05 | 53.5 | — | |
| Physics ReasonerBackbone=GPT-4o2025.05 | 36.6 | — | |
| Creator2025.05 | — | 59.5 | |
| Physics Reasoner2025.05 | — | 53.9 | |
| Physics ReasonerFew-shot=4-shot2025.05 | — | 67.6 | |
| PoT2025.05 | — | 63.7 | |
| PoT + RAG2025.05 | — | 65.1 | |
| PoT + REFTOOL2025.05 | — | 67.6 |