Information Retention on ROBENCH-25 1.0 (test)
100CHSS (No Trigger)Jamba-1.7-Mini
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Jamba-1.7-MiniAwareness Instruction=false, Recovery Instruction=false2026.01 | 100 | 100 | 99.2 | 99.2 | 100 | 100 | 100 | 100 | |
| Jamba-1.7-MiniAwareness Instruction=false, Recovery Instruction=true2026.01 | 99.2 | 99.2 | 99.2 | 99.2 | 99.2 | 99.2 | 99.2 | 99.2 | |
| Jamba-1.7-MiniAwareness Instruction=true, Recovery Instruction=false2026.01 | 99.2 | 99.2 | 99.2 | 99.2 | 99.2 | 99.2 | 99.2 | 99.2 | |
| Jamba-1.7-MiniAwareness Instruction=true, Recovery Instruction=true2026.01 | 99.2 | 99.2 | 99.2 | 99.2 | 99.2 | 99.2 | 99.2 | 99.2 | |
| Mamba-2.8bAwareness Instruction=false, Recovery Instruction=true2026.01 | 27.5 | 21.7 | 8.3 | 13.3 | 15 | 15 | 15 | 0 | |
| Mamba-2.8bAwareness Instruction=false, Recovery Instruction=false2026.01 | 24.2 | 20 | 4.2 | 10 | 1.7 | 6.7 | 2.5 | 0 | |
| Mamba-2.8bAwareness Instruction=true, Recovery Instruction=true2026.01 | 20.8 | 17.5 | 7.5 | 13.3 | 7.5 | 10.8 | 8.3 | 0 | |
| Mamba-2.8bAwareness Instruction=true, Recovery Instruction=false2026.01 | 19.2 | 17.5 | 6.7 | 15 | 7.5 | 14.2 | 10 | 0 | |
| Pythia-2.8bAwareness Instruction=false, Recovery Instruction=true2026.01 | 5.8 | 3.3 | 2.9 | 21.7 | 0 | 19.2 | 5 | 7.9 | |
| Pythia-2.8bAwareness Instruction=false, Recovery Instruction=false2026.01 | 3.3 | 7.1 | 4.2 | 4.2 | 0.4 | 13.8 | 5.4 | 1.7 | |
| Pythia-2.8bAwareness Instruction=true, Recovery Instruction=false2026.01 | 1.3 | 3.3 | 0 | 0 | 0 | 0.8 | 0 | 0 | |
| Pythia-2.8bAwareness Instruction=true, Recovery Instruction=true2026.01 | 0 | 0 | 0 | 7.1 | 0 | 3.8 | 0 | 0 |