Monitoring on MonitoringBench Full Trajectory 1.0
94.8FW ScoreOpus 4.5
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Opus 4.5Pipeline=Baseline, n (FW/MW/PW/Sudo)=58/42/43/532026.05 | 94.8 | 97.6 | 95.3 | 92.5 | |
| Opus 4.5Pipeline=Human Strategies, n (FW/MW/PW/Sudo)=256/101/110/3132026.05 | 91.3 | 100 | 99.4 | 93.5 | |
| Sonnet 4.5Pipeline=Baseline, n (FW/MW/PW/Sudo)=58/42/43/532026.05 | 77.6 | 95.2 | 83.7 | 77.4 | |
| Opus 4.5Pipeline=Refined, n (FW/MW/PW/Sudo)=511/250/198/4772026.05 | 75.9 | 87 | 43.8 | 65.8 | |
| Sonnet 4.5Pipeline=Human Strategies, n (FW/MW/PW/Sudo)=256/101/110/3132026.05 | 74.5 | 99 | 98.2 | 78.4 | |
| Opus 4.5Pipeline=Recon, n (FW/MW/PW/Sudo)=204/669/128/2652026.05 | 72.9 | 85.1 | 75.8 | 65.9 | |
| Sonnet 4.0Pipeline=Baseline, n (FW/MW/PW/Sudo)=58/42/43/532026.05 | 67.2 | 71.4 | 48.8 | 47.2 | |
| Sonnet 4.0Pipeline=Human Strategies, n (FW/MW/PW/Sudo)=256/101/110/3132026.05 | 66.9 | 94.7 | 84.5 | 62.9 | |
| Sonnet 4.5Pipeline=Recon, n (FW/MW/PW/Sudo)=204/669/128/2652026.05 | 62.7 | 76.6 | 68 | 38.4 | |
| GPT-5.1 CodexPipeline=Human Strategies, n (FW/MW/PW/Sudo)=256/101/110/3132026.05 | 60.6 | 100 | 98.4 | 84 | |
| Sonnet 4.0Pipeline=Recon, n (FW/MW/PW/Sudo)=204/669/128/2652026.05 | 56 | 59.5 | 49.2 | 30.9 | |
| Sonnet 4.5Pipeline=Refined, n (FW/MW/PW/Sudo)=511/250/198/4772026.05 | 52 | 68.5 | 36.2 | 35.8 | |
| GPT-5.1 CodexPipeline=Recon, n (FW/MW/PW/Sudo)=204/669/128/2652026.05 | 51.8 | 68.2 | 68.2 | 45.6 | |
| GPT-5.1 CodexPipeline=Baseline, n (FW/MW/PW/Sudo)=58/42/43/532026.05 | 50 | 85 | 65.1 | 84.3 | |
| GPT-5.1 CodexPipeline=Refined, n (FW/MW/PW/Sudo)=511/250/198/4772026.05 | 46.5 | 37.3 | 38.6 | 31.8 | |
| Sonnet 4.0Pipeline=Refined, n (FW/MW/PW/Sudo)=511/250/198/4772026.05 | 44 | 38.5 | 22.7 | 29.4 | |
| o3Pipeline=Recon, n (FW/MW/PW/Sudo)=204/669/128/2652026.05 | 35.9 | 67.5 | 62.5 | 29.3 | |
| GPT-5Pipeline=Recon, n (FW/MW/PW/Sudo)=204/669/128/2652026.05 | 29.7 | 68.7 | 64.1 | 27.7 | |
| o3Pipeline=Baseline, n (FW/MW/PW/Sudo)=58/42/43/532026.05 | 24.1 | 88.1 | 60.5 | 84.9 | |
| o3Pipeline=Human Strategies, n (FW/MW/PW/Sudo)=256/101/110/3132026.05 | 22.8 | 96 | 92.1 | 66.3 | |
| GPT-5Pipeline=Baseline, n (FW/MW/PW/Sudo)=58/42/43/532026.05 | 22.4 | 85.7 | 65.1 | 77.4 | |
| GPT-5Pipeline=Human Strategies, n (FW/MW/PW/Sudo)=256/101/110/3132026.05 | 16.7 | 98.7 | 95.5 | 70.2 | |
| GPT-4.1Pipeline=Recon, n (FW/MW/PW/Sudo)=204/669/128/2652026.05 | 16.5 | 29.1 | 31.8 | 10.9 | |
| GPT-4.1Pipeline=Human Strategies, n (FW/MW/PW/Sudo)=256/101/110/3132026.05 | 13.3 | 74.6 | 68.8 | 30.4 | |
| GPT-4 TurboPipeline=Human Strategies, n (FW/MW/PW/Sudo)=256/101/110/3132026.05 | 13.2 | 43.9 | 29.4 | 14.2 | |
| GPT-4oPipeline=Recon, n (FW/MW/PW/Sudo)=204/669/128/2652026.05 | 12.7 | 22.1 | 12.5 | 11.7 | |
| GPT-4oPipeline=Human Strategies, n (FW/MW/PW/Sudo)=256/101/110/3132026.05 | 11.6 | 27.7 | 31.3 | 15.1 | |
| o3Pipeline=Refined, n (FW/MW/PW/Sudo)=511/250/198/4772026.05 | 10.8 | 32.3 | 22.4 | 13.9 | |
| GPT-4 TurboPipeline=Recon, n (FW/MW/PW/Sudo)=204/669/128/2652026.05 | 7.5 | 19.4 | 9.9 | 8.1 | |
| GPT-4.1Pipeline=Refined, n (FW/MW/PW/Sudo)=511/250/198/4772026.05 | 7.3 | 3.2 | 4.5 | 2.2 | |
| GPT-5Pipeline=Refined, n (FW/MW/PW/Sudo)=511/250/198/4772026.05 | 5.8 | 18.6 | 28.1 | 16.9 | |
| GPT-5 NanoPipeline=Recon, n (FW/MW/PW/Sudo)=204/669/128/2652026.05 | 5.7 | 11.2 | 9.2 | 9.3 | |
| GPT-4.1Pipeline=Baseline, n (FW/MW/PW/Sudo)=58/42/43/532026.05 | 5.2 | 19 | 14 | 5.7 | |
| GPT-4 TurboPipeline=Baseline, n (FW/MW/PW/Sudo)=58/42/43/532026.05 | 5.2 | 0 | 4.7 | 0 | |
| GPT-4 TurboPipeline=Refined, n (FW/MW/PW/Sudo)=511/250/198/4772026.05 | 4.9 | 3.7 | 2.2 | 1.1 | |
| GPT-4.1 MiniPipeline=Recon, n (FW/MW/PW/Sudo)=204/669/128/2652026.05 | 4.7 | 8.8 | 9.6 | 3.3 | |
| GPT-4oPipeline=Refined, n (FW/MW/PW/Sudo)=511/250/198/4772026.05 | 4.4 | 2.1 | 1.9 | 1.8 | |
| GPT-4oPipeline=Baseline, n (FW/MW/PW/Sudo)=58/42/43/532026.05 | 3.4 | 14.3 | 7 | 5.7 | |
| GPT-4.1 MiniPipeline=Human Strategies, n (FW/MW/PW/Sudo)=256/101/110/3132026.05 | 2.9 | 17.8 | 31.5 | 5 | |
| o4-miniPipeline=Recon, n (FW/MW/PW/Sudo)=204/669/128/2652026.05 | 2.5 | 15.9 | 11 | 3.2 | |
| GPT-5 NanoPipeline=Baseline, n (FW/MW/PW/Sudo)=58/42/43/532026.05 | 1.7 | 2.4 | 0 | 17 | |
| GPT-4.1 MiniPipeline=Refined, n (FW/MW/PW/Sudo)=511/250/198/4772026.05 | 1.5 | 1.1 | 0.3 | 0.3 | |
| o4-miniPipeline=Human Strategies, n (FW/MW/PW/Sudo)=256/101/110/3132026.05 | 0.5 | 50.3 | 36.7 | 11.1 | |
| GPT-4o MiniPipeline=Recon, n (FW/MW/PW/Sudo)=204/669/128/2652026.05 | 0.5 | 1.1 | 0 | 0.8 | |
| GPT-5 NanoPipeline=Human Strategies, n (FW/MW/PW/Sudo)=256/101/110/3132026.05 | 0.4 | 20.9 | 21.3 | 23.4 | |
| GPT-5 NanoPipeline=Refined, n (FW/MW/PW/Sudo)=511/250/198/4772026.05 | 0.3 | 0.9 | 0.7 | 3.4 | |
| o4-miniPipeline=Refined, n (FW/MW/PW/Sudo)=511/250/198/4772026.05 | 0.1 | 0.3 | 0.3 | 0.3 | |
| GPT-4o MiniPipeline=Human Strategies, n (FW/MW/PW/Sudo)=256/101/110/3132026.05 | 0.1 | 2 | 0.9 | 0.7 | |
| GPT-4o MiniPipeline=Refined, n (FW/MW/PW/Sudo)=511/250/198/4772026.05 | 0.1 | 0 | 0 | 0.5 | |
| o4-miniPipeline=Baseline, n (FW/MW/PW/Sudo)=58/42/43/532026.05 | 0 | 2.4 | 4.7 | 5.7 | |
| GPT-4.1 MiniPipeline=Baseline, n (FW/MW/PW/Sudo)=58/42/43/532026.05 | 0 | 0 | 0 | 1.9 | |
| GPT-4o MiniPipeline=Baseline, n (FW/MW/PW/Sudo)=58/42/43/532026.05 | 0 | 0 | 0 | 0 |