Data Operations on QGP-DataOps lite (full)
50Success RateUNITQGP
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| UNITQGPModel=gpt-4.1-mini, Backend=Native, Backlogs=242026.05 | 50 | 8 | 5.9 | 13.6 | 50 | 0 | 0 | |
| UNITQGPModel=gpt-4.1-mini, Backend=LangGraph, Backlogs=242026.05 | 50 | 8.042 | 10.1 | 14.4 | 50 | 0 | 0 | |
| StandardModel=gpt-4.1, Backend=LETTA/MEMGPT, Backlogs=242026.05 | 50 | 7.5 | 4.9 | 24.5 | 0 | 50 | 0 | |
| StandardModel=gpt-5.4, Backend=LETTA/MEMGPT, Backlogs=242026.05 | 45.8 | 7.042 | 4 | 29 | 0 | 54.2 | 0 | |
| StandardModel=gpt-5.4, Backend=LG+Mem, Backlogs=242026.05 | 41.7 | 7.208 | 1.8 | 29.9 | 37.5 | 20.8 | 0 | |
| UNITQGPModel=gpt-4.1, Backend=Native, Backlogs=242026.05 | 29.2 | 7.333 | 56.9 | 10.2 | 70.8 | 0 | 0 | |
| UNITQGPModel=gpt-4.1, Backend=LangGraph, Backlogs=242026.05 | 29.2 | 7.167 | 60.4 | 9.8 | 70.8 | 0 | 0 | |
| UNITQGPModel=gpt-5.4, Backend=Native, Backlogs=242026.05 | 25 | 6.708 | 0 | 9.4 | 75 | 0 | 0 | |
| UNITQGPModel=gpt-5.4, Backend=LangGraph, Backlogs=242026.05 | 25 | 6.542 | 6 | 9.2 | 75 | 0 | 0 | |
| StandardModel=gpt-4.1-mini, Backend=LETTA/MEMGPT, Backlogs=242026.05 | 20.8 | 4.917 | 4.4 | 24.2 | 0 | 79.2 | 0 | |
| StandardModel=gpt-4.1, Backend=LG+Mem, Backlogs=242026.05 | 8.3 | 5.833 | 1.3 | 19 | 25 | 50 | 16.7 | |
| StandardModel=gpt-4.1-mini, Backend=LG+Mem, Backlogs=242026.05 | 4.2 | 2.25 | 25.3 | 5.4 | 20.8 | 58.3 | 16.7 | |
| StandardModel=gpt-4.1-mini, Backend=Native, Backlogs=242026.05 | 0 | 0.708 | 12.5 | 3.6 | 83.3 | 16.7 | 0 | |
| VGModel=gpt-4.1-mini, Backend=Native, Backlogs=242026.05 | 0 | 0.833 | 8.3 | 3.7 | 100 | 0 | 0 | |
| StandardModel=gpt-4.1-mini, Backend=LangGraph, Backlogs=242026.05 | 0 | 0.708 | 5 | 4.6 | 91.7 | 8.3 | 0 | |
| VGModel=gpt-4.1-mini, Backend=LangGraph, Backlogs=242026.05 | 0 | 0.792 | 4.2 | 4.2 | 100 | 0 | 0 | |
| StandardModel=gpt-4.1, Backend=Native, Backlogs=242026.05 | 0 | 0.875 | 48.3 | 4.6 | 79.2 | 20.8 | 0 | |
| VGModel=gpt-4.1, Backend=Native, Backlogs=242026.05 | 0 | 2.417 | 21.6 | 8.9 | 100 | 0 | 0 | |
| StandardModel=gpt-4.1, Backend=LangGraph, Backlogs=242026.05 | 0 | 1.125 | 55.8 | 5.7 | 79.2 | 20.8 | 0 | |
| VGModel=gpt-4.1, Backend=LangGraph, Backlogs=242026.05 | 0 | 2.625 | 42.7 | 6.9 | 100 | 0 | 0 | |
| StandardModel=gpt-5.4, Backend=Native, Backlogs=242026.05 | 0 | 0 | 0 | 0 | 54.2 | 45.8 | 0 | |
| VGModel=gpt-5.4, Backend=Native, Backlogs=242026.05 | 0 | 0 | 0 | 0 | 100 | 0 | 0 | |
| StandardModel=gpt-5.4, Backend=LangGraph, Backlogs=242026.05 | 0 | 0 | 0 | 0 | 100 | 0 | 0 | |
| VGModel=gpt-5.4, Backend=LangGraph, Backlogs=242026.05 | 0 | 0 | 0 | 0 | 100 | 0 | 0 |