Data Discovery and Query Solving on KramaBench Original Setting 2025
41.67Archaeology ScoreData Interpreter
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Data InterpreterFramework=Data Interpreter, Model=Gemini-2.5-Pro2025.09 | 41.67 | 12.72 | 28.05 | 9.87 | 30.04 | 59.67 | 31.32 | |
| DA-AgentFramework=DA-Agent, Model=Gemini-2.5-Pro2025.09 | 41.67 | 15.52 | 12.59 | 42.64 | 39.73 | 61.61 | 39.79 | |
| o3Framework=Model-only, Model=o32025.09 | 25 | 1.73 | 3.5 | 1.35 | 3.35 | 24.87 | 9.64 | |
| DS-GURUFramework=DS-GURU, Model=o32025.09 | 25 | 3.53 | 8.95 | 19.6 | 13.89 | 50.73 | 22.08 | |
| DS-STARFramework=DS-STAR, Model=Gemini-2.5-Pro2025.09 | 25 | 12.09 | 43.74 | 46.75 | 49.64 | 65.94 | 44.69 | |
| Claude-3.5-SonnetFramework=Model-only, Model=Claude-3.5-Sonnet2025.09 | 16.67 | 1.62 | 2.87 | 1.17 | 7.33 | 13.63 | 7.45 | |
| DS-GURUFramework=DS-GURU, Model=GPT-4o2025.09 | 16.67 | 2.76 | 8.97 | 2.6 | 2.8 | 17.18 | 8.28 | |
| DS-GURUFramework=DS-GURU, Model=Claude-3.5-Sonnet2025.09 | 16.67 | 1.52 | 1.96 | 11.21 | 7.01 | 39.16 | 14.35 | |
| ReActFramework=ReAct, Model=Gemini-2.5-Pro2025.09 | 16.67 | 4.77 | 3.69 | 26.17 | 41.95 | 51.4 | 30.31 | |
| AutoGenFramework=AutoGen, Model=Gemini-2.5-Pro2025.09 | 16.67 | 4.39 | 7.25 | 19.38 | 26.38 | 41.76 | 22.83 | |
| GPT-4oFramework=Model-only, Model=GPT-4o2025.09 | 0 | 1.41 | 1.98 | 0.45 | 1.46 | 1.45 | 1.62 |