Cyber Defense Mean Return on CybORG CAGE-2 Post-Session Evaluation (test)
-14Mean ReturnFORGE w/o grad
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| FORGE w/o gradModel=Grok, Representation=Examples2026.05 | -14 | 2 | 76 | 78 | 67 | |
| FORGE w/o gradModel=Qwen, Representation=Examples2026.05 | -15.5 | 2.3 | 85 | 73 | 36 | |
| FORGE w/o gradModel=Qwen, Representation=Rules2026.05 | -17.3 | 2.6 | 83 | 80 | 31 | |
| FORGE w/o gradModel=Qwen, Representation=Mixed2026.05 | -17.9 | 2.8 | 83 | 78 | 39 | |
| FORGE w/o gradModel=Grok, Representation=Mixed2026.05 | -23.2 | 19.4 | 60 | 80 | 45 | |
| FORGE w/o gradModel=Llama, Representation=Mixed2026.05 | -23.8 | 9.3 | 79 | 46 | 19 | |
| FORGEModel=Qwen, Representation=Examples2026.05 | -24.3 | 35.8 | 77 | 58 | — | |
| FORGEModel=Gemini, Representation=Examples2026.05 | -24.5 | 21.1 | 87 | 69 | — | |
| FORGE w/o gradModel=Grok, Representation=Rules2026.05 | -24.5 | 15.1 | 58 | 69 | 27 | |
| FORGEModel=Qwen, Representation=Rules2026.05 | -25.2 | 21 | 76 | 72 | — | |
| FORGEModel=Llama, Representation=Examples2026.05 | -28.3 | 15.7 | 75 | 48 | — | |
| FORGEModel=Qwen, Representation=Mixed2026.05 | -29.3 | 20.4 | 72 | 64 | — | |
| FORGEModel=Llama, Representation=Mixed2026.05 | -29.6 | 25.6 | 74 | 33 | — | |
| FORGEModel=Gemini, Representation=Rules2026.05 | -30.6 | 37 | 84 | 51 | — | |
| FORGEModel=Gemini, Representation=Mixed2026.05 | -32.2 | 28.1 | 83 | 61 | — | |
| FORGE w/o gradModel=Gemini, Representation=Mixed2026.05 | -32.7 | 30.7 | 83 | 60 | 1 | |
| FORGE w/o gradModel=Gemini, Representation=Rules2026.05 | -33.1 | 26.5 | 83 | 47 | 8 | |
| FORGEModel=Grok, Representation=Rules2026.05 | -33.7 | 26.1 | 42 | 58 | — | |
| FORGE w/o gradModel=Gemini, Representation=Examples2026.05 | -37.7 | 27.2 | 80 | 52 | 54 | |
| FORGEModel=Grok, Representation=Mixed2026.05 | -42.2 | 36.9 | 28 | 63 | — | |
| FORGE w/o gradModel=Llama, Representation=Examples2026.05 | -42.2 | 35.3 | 63 | 22 | 49 | |
| FORGEModel=Grok, Representation=Examples2026.05 | -42.7 | 43.3 | 27 | 34 | — | |
| ReflexionModel=Llama, Representation=Mixed2026.05 | -44.2 | 40.6 | 61 | — | — | |
| ReflexionModel=Llama, Representation=Examples2026.05 | -53.9 | 59.7 | 52 | — | — | |
| ReflexionModel=Qwen, Representation=Examples2026.05 | -57.6 | 69.5 | 44 | — | — | |
| Zero-ShotModel=Grok2026.05 | -58.4 | 55.2 | — | — | — | |
| ReflexionModel=Gemini, Representation=Rules2026.05 | -62.7 | 60.5 | 67 | — | — | |
| ReflexionModel=Grok, Representation=Examples2026.05 | -64.8 | 52.4 | 11 | — | — | |
| FORGEModel=Llama, Representation=Rules2026.05 | -72 | 46.6 | 36 | 29 | — | |
| FORGE w/o gradModel=Llama, Representation=Rules2026.05 | -76.7 | 54.7 | 32 | 24 | 7 | |
| ReflexionModel=Gemini, Representation=Examples2026.05 | -78.9 | 60.4 | 58 | — | — | |
| ReflexionModel=Grok, Representation=Rules2026.05 | -79.9 | 62.6 | 37 | — | — | |
| ReflexionModel=Qwen, Representation=Mixed2026.05 | -80.4 | 89.4 | 22 | — | — | |
| ReflexionModel=Gemini, Representation=Mixed2026.05 | -81.9 | 74.6 | 57 | — | — | |
| ReflexionModel=Qwen, Representation=Rules2026.05 | -88.4 | 83.5 | 14 | — | — | |
| ReflexionModel=Llama, Representation=Rules2026.05 | -101.4 | 61 | 10 | — | — | |
| Zero-ShotModel=Qwen2026.05 | -103.3 | 87.3 | — | — | — | |
| Zero-ShotModel=Llama2026.05 | -113.1 | 81.5 | — | — | — | |
| ReflexionModel=Grok, Representation=Mixed2026.05 | -114.4 | 76.8 | 96 | — | — | |
| Zero-ShotModel=Gemini2026.05 | -189.6 | 53.9 | — | — | — |