Learning and Mastery in Text-Adventure Games on J-TTL Jericho suite (test)
0.93Detective AUCEvoTest
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| EvoTestBackbone=google/gemini-2.5-flash, Number of seeds=5, Learning paradigm=Evolutionary Optimization2025.10 | 0.93 | 0.78 | 0.15 | 0.36 | 0.33 | 0.32 | 0.48 | |
| PromptbreederBackbone=google/gemini-2.5-flash, Number of seeds=5, Learning paradigm=Automated Prompt Optimization Methods2025.10 | 0.64 | 0.48 | 0.11 | 0.3 | 0.24 | 0.31 | 0.35 | |
| EvoPromptBackbone=google/gemini-2.5-flash, Number of seeds=5, Learning paradigm=Automated Prompt Optimization Methods2025.10 | 0.64 | 0.48 | 0.13 | 0.31 | 0.25 | 0.28 | 0.35 | |
| TextGradBackbone=google/gemini-2.5-flash, Number of seeds=5, Learning paradigm=Automated Prompt Optimization Methods2025.10 | 0.62 | 0.46 | 0.12 | 0.29 | 0.17 | 0.24 | 0.32 | |
| ReflexionBackbone=google/gemini-2.5-flash, Number of seeds=5, Learning paradigm=Memory-based & Reflection-based Methods2025.10 | 0.59 | 0.42 | 0.09 | 0.26 | 0.31 | 0.3 | 0.33 | |
| GRPO (online)Backbone=google/gemini-2.5-flash, Number of seeds=5, Learning paradigm=Weight-Update Methods (Online Fine-Tuning)2025.10 | 0.58 | 0.39 | 0.08 | 0.23 | 0.31 | 0.25 | 0.31 | |
| MemoryBackbone=google/gemini-2.5-flash, Number of seeds=5, Learning paradigm=Memory-based & Reflection-based Methods2025.10 | 0.56 | 0.19 | 0.04 | 0.06 | 0.14 | 0.11 | 0.13 | |
| SummaryBackbone=google/gemini-2.5-flash, Number of seeds=5, Learning paradigm=Memory-based & Reflection-based Methods2025.10 | 0.46 | 0.34 | 0.11 | 0.16 | 0.26 | 0.22 | 0.26 | |
| SFT (online)Backbone=google/gemini-2.5-flash, Number of seeds=5, Learning paradigm=Weight-Update Methods (Online Fine-Tuning)2025.10 | 0.41 | 0.31 | 0.11 | 0.19 | 0.23 | 0.2 | 0.24 | |
| RAGBackbone=google/gemini-2.5-flash, Number of seeds=5, Learning paradigm=Memory-based & Reflection-based Methods2025.10 | 0.35 | 0.25 | 0.07 | 0.1 | 0.19 | 0.16 | 0.18 | |
| StaticBackbone=google/gemini-2.5-flash, Number of seeds=5, Learning paradigm=Non-learning Baseline2025.10 | 0.22 | 0.14 | 0.03 | 0.05 | 0.11 | 0.08 | 0.11 |