Chemistry Evaluation on SciBench
66.4AccuracyPoT + REFTOOL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| PoT + REFTOOLModel Backbone=Gemini-1.5-Pro2025.05 | 66.4 | — | |
| TroVEModel Backbone=Gemini-1.5-Pro2025.05 | 65.6 | — | |
| ChemAgentModel Backbone=Gemini-1.5-Pro2025.05 | 65.5 | — | |
| PoT + RAGModel Backbone=Gemini-1.5-Pro2025.05 | 63.7 | — | |
| PoTModel Backbone=Gemini-1.5-Pro2025.05 | 62.3 | — | |
| PoT + REFTOOLModel Backbone=GPT-4o2025.05 | 61.3 | — | |
| CreatorModel Backbone=Gemini-1.5-Pro2025.05 | 60 | — | |
| ChemAgentModel Backbone=GPT-4o2025.05 | 58.9 | — | |
| PoTModel Backbone=GPT-4o2025.05 | 58.9 | — | |
| PoT + RAGModel Backbone=GPT-4o2025.05 | 56.6 | — | |
| PoT + RAGModel Backbone=GPT-42025.05 | 54.1 | — | |
| PoT + REFTOOLModel Backbone=GPT-42025.05 | 53.4 | — | |
| TroVEModel Backbone=GPT-4o2025.05 | 52.7 | — | |
| ChemAgentModel Backbone=GPT-42025.05 | 52.5 | — | |
| PoTModel Backbone=GPT-42025.05 | 51.8 | — | |
| StructChemModel Backbone=Gemini-1.5-Pro2025.05 | 50.2 | — | |
| PoT + REFTOOLModel Backbone=Llama-3.1-70B2025.05 | 49.5 | — | |
| ChemAgentModel Backbone=Llama-3.1-70B2025.05 | 48.2 | — | |
| PoT + RAGModel Backbone=Llama-3.1-70B2025.05 | 48.1 | — | |
| CreatorModel Backbone=GPT-42025.05 | 46.9 | — | |
| PoTModel Backbone=Llama-3.1-70B2025.05 | 46.9 | — | |
| LATMModel Backbone=GPT-42025.05 | 45 | — | |
| CreatorModel Backbone=GPT-4o2025.05 | 43.3 | — | |
| StructChemModel Backbone=GPT-4o2025.05 | 40.5 | — | |
| CreatorModel Backbone=Llama-3.1-70B2025.05 | 40.1 | — | |
| TroVEModel Backbone=GPT-42025.05 | 39.2 | — | |
| TroVEModel Backbone=Llama-3.1-70B2025.05 | 38.6 | — | |
| StructChemModel Backbone=Llama-3.1-70B2025.05 | 37.9 | — | |
| LATMModel Backbone=GPT-4o2025.05 | 35.7 | — | |
| LATMModel Backbone=Llama-3.1-70B2025.05 | 31.4 | — | |
| StructChemModel Backbone=GPT-42025.05 | 29.7 | — | |
| LATMModel Backbone=Gemini-1.5-Pro2025.05 | 25.1 | — | |
| ChemAgent2025.05 | — | 56.3 | |
| Creator2025.05 | — | 47.6 | |
| LATM2025.05 | — | 34.3 | |
| PoT2025.05 | — | 55 | |
| PoT + RAG2025.05 | — | 55.6 | |
| PoT + REFTOOL2025.05 | — | 57.7 | |
| StructChem2025.05 | — | 39.6 | |
| TroVE2025.05 | — | 49 |