Physics on TheoremQA
58.8AccuracyTroVE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| TroVEModel Backbone=Gemini-1.5-Pro2025.05 | 58.8 | — | |
| PoT + REFTOOLModel Backbone=Gemini-1.5-Pro2025.05 | 58.8 | — | |
| PoTModel Backbone=Gemini-1.5-Pro2025.05 | 57.9 | — | |
| PoT + RAGModel Backbone=GPT-4o2025.05 | 57.9 | — | |
| PoT + REFTOOLModel Backbone=GPT-4o2025.05 | 57.9 | — | |
| CreatorModel Backbone=Gemini-1.5-Pro2025.05 | 57 | — | |
| PoTModel Backbone=GPT-4o2025.05 | 57 | — | |
| PoT + RAGModel Backbone=Gemini-1.5-Pro2025.05 | 57 | — | |
| PoT + REFTOOLModel Backbone=Llama-3.1-70B2025.05 | 53.5 | — | |
| Physics ReasonerModel Backbone=Gemini-1.5-Pro2025.05 | 50.9 | — | |
| PoT + REFTOOLModel Backbone=GPT-42025.05 | 49.1 | — | |
| TroVEModel Backbone=GPT-4o2025.05 | 48.2 | — | |
| Physics ReasonerModel Backbone=Llama-3.1-70B2025.05 | 48.2 | — | |
| PoTModel Backbone=Llama-3.1-70B2025.05 | 48.2 | — | |
| PoTModel Backbone=GPT-42025.05 | 45.6 | — | |
| PoT + RAGModel Backbone=Llama-3.1-70B2025.05 | 44.7 | — | |
| PoT + RAGModel Backbone=GPT-42025.05 | 44.7 | — | |
| Physics ReasonerModel Backbone=GPT-42025.05 | 42.1 | — | |
| CreatorModel Backbone=Llama-3.1-70B2025.05 | 40.4 | — | |
| CreatorModel Backbone=GPT-4o2025.05 | 40.4 | — | |
| LATMModel Backbone=GPT-42025.05 | 39.8 | — | |
| LATMModel Backbone=Llama-3.1-70B2025.05 | 38.9 | — | |
| CreatorModel Backbone=GPT-42025.05 | 35.1 | — | |
| TroVEModel Backbone=GPT-42025.05 | 35.1 | — | |
| LATMModel Backbone=Gemini-1.5-Pro2025.05 | 33.3 | — | |
| TroVEModel Backbone=Llama-3.1-70B2025.05 | 33.3 | — | |
| Physics ReasonerModel Backbone=GPT-4o2025.05 | 33.3 | — | |
| LATMModel Backbone=GPT-4o2025.05 | 30.6 | — | |
| Creator2025.05 | — | 43.2 | |
| LATM2025.05 | — | 35.7 | |
| Physics Reasoner2025.05 | — | 43.6 | |
| PoT2025.05 | — | 52.2 | |
| PoT + RAG2025.05 | — | 51.1 | |
| PoT + REFTOOL2025.05 | — | 54.8 | |
| TroVE2025.05 | — | 43.9 |