Honest commitment prediction on Commitment prediction dataset (Leave-one-environment-out OOD split)
0.729Mean OOD AUROCAttention + PCA final
Evaluation results
| Method | Links | |
|---|---|---|
| Attention + PCA finalModel=GPT-OSS-20B, Feature Category=Combined2026.05 | 0.729 | |
| Attention + PCA final − mean(prev 4)Model=R1-Distill Qwen-14B, Feature Category=Combined2026.05 | 0.712 | |
| All attentionModel=GPT-OSS-20B, Feature Category=Attention2026.05 | 0.711 | |
| Attention + PCA final − prevModel=GPT-OSS-20B, Feature Category=Combined2026.05 | 0.711 | |
| Attention + PCA final − mean(prev 4)Model=GPT-OSS-20B, Feature Category=Combined2026.05 | 0.709 | |
| Grounding transition onlyModel=GPT-OSS-20B, Feature Category=Attention2026.05 | 0.706 | |
| All attentionModel=R1-Distill Qwen-14B, Feature Category=Attention2026.05 | 0.705 | |
| Attention + PCA final − prevModel=R1-Distill Qwen-14B, Feature Category=Combined2026.05 | 0.7 | |
| Attention + PCA finalModel=R1-Distill Qwen-14B, Feature Category=Combined2026.05 | 0.695 | |
| Grounding transition onlyModel=R1-Distill Qwen-14B, Feature Category=Attention2026.05 | 0.691 | |
| Attention + PCA finalModel=R1-Distill Llama-8B, Feature Category=Combined2026.05 | 0.689 | |
| RawModel=R1-Distill Qwen-7B, Feature Category=Activation2026.05 | 0.683 | |
| Attention + PCA finalModel=R1-Distill Qwen-7B, Feature Category=Combined2026.05 | 0.68 | |
| Attention + PCA final − prevModel=R1-Distill Llama-8B, Feature Category=Combined2026.05 | 0.673 | |
| Attention + PCA final − mean(prev 4)Model=R1-Distill Llama-8B, Feature Category=Combined2026.05 | 0.668 | |
| Attention + PCA final − mean(prev 4)Model=R1-Distill Qwen-7B, Feature Category=Combined2026.05 | 0.667 | |
| All attentionModel=R1-Distill Qwen-7B, Feature Category=Attention2026.05 | 0.666 | |
| PCA finalModel=R1-Distill Llama-8B, Feature Category=Activation2026.05 | 0.664 | |
| Grounding onlyModel=GPT-OSS-20B, Feature Category=Attention2026.05 | 0.664 | |
| RawModel=R1-Distill Llama-8B, Feature Category=Activation2026.05 | 0.662 | |
| Grounding transition onlyModel=R1-Distill Qwen-7B, Feature Category=Attention2026.05 | 0.661 | |
| All attentionModel=R1-Distill Llama-8B, Feature Category=Attention2026.05 | 0.66 | |
| Attention + PCA final − prevModel=R1-Distill Qwen-7B, Feature Category=Combined2026.05 | 0.659 | |
| Grounding onlyModel=R1-Distill Qwen-14B, Feature Category=Attention2026.05 | 0.656 | |
| PCA final − mean(prev 4)Model=R1-Distill Qwen-14B, Feature Category=Activation2026.05 | 0.653 | |
| Concentration onlyModel=R1-Distill Qwen-7B, Feature Category=Attention2026.05 | 0.649 | |
| Grounding onlyModel=R1-Distill Qwen-7B, Feature Category=Attention2026.05 | 0.645 | |
| Concentration onlyModel=R1-Distill Qwen-14B, Feature Category=Attention2026.05 | 0.645 | |
| PCA finalModel=R1-Distill Qwen-7B, Feature Category=Activation2026.05 | 0.644 | |
| Concentration trans. onlyModel=R1-Distill Qwen-14B, Feature Category=Attention2026.05 | 0.644 | |
| RawModel=R1-Distill Qwen-14B, Feature Category=Activation2026.05 | 0.643 | |
| Grounding transition onlyModel=R1-Distill Llama-8B, Feature Category=Attention2026.05 | 0.637 | |
| PCA final − mean(prev 4)Model=R1-Distill Llama-8B, Feature Category=Activation2026.05 | 0.635 | |
| PCA finalModel=R1-Distill Qwen-14B, Feature Category=Activation2026.05 | 0.628 | |
| PCA final − mean(prev 4)Model=R1-Distill Qwen-7B, Feature Category=Activation2026.05 | 0.628 | |
| PCA final − prevModel=R1-Distill Llama-8B, Feature Category=Activation2026.05 | 0.626 | |
| Concentration trans. onlyModel=R1-Distill Qwen-7B, Feature Category=Attention2026.05 | 0.626 | |
| Concentration onlyModel=GPT-OSS-20B, Feature Category=Attention2026.05 | 0.625 | |
| Concentration trans. onlyModel=GPT-OSS-20B, Feature Category=Attention2026.05 | 0.624 | |
| PCA final − prevModel=R1-Distill Qwen-7B, Feature Category=Activation2026.05 | 0.623 | |
| PCA final − prevModel=R1-Distill Qwen-14B, Feature Category=Activation2026.05 | 0.62 | |
| Concentration trans. onlyModel=R1-Distill Llama-8B, Feature Category=Attention2026.05 | 0.615 | |
| PCA finalModel=GPT-OSS-20B, Feature Category=Activation2026.05 | 0.612 | |
| PCA final − mean(prev 4)Model=GPT-OSS-20B, Feature Category=Activation2026.05 | 0.605 | |
| Concentration onlyModel=R1-Distill Llama-8B, Feature Category=Attention2026.05 | 0.604 | |
| RawModel=GPT-OSS-20B, Feature Category=Activation2026.05 | 0.601 | |
| PCA final − prevModel=GPT-OSS-20B, Feature Category=Activation2026.05 | 0.598 | |
| Grounding onlyModel=R1-Distill Llama-8B, Feature Category=Attention2026.05 | 0.598 | |
| TF-IDF last sentenceModel=R1-Distill Qwen-7B, Feature Category=Lexical Baselines2026.05 | 0.562 | |
| TF-IDF prefixModel=R1-Distill Qwen-7B, Feature Category=Lexical Baselines2026.05 | 0.546 | |
| TF-IDF last sentenceModel=R1-Distill Llama-8B, Feature Category=Lexical Baselines2026.05 | 0.516 | |
| TF-IDF last sentenceModel=GPT-OSS-20B, Feature Category=Lexical Baselines2026.05 | 0.501 | |
| TF-IDF prefixModel=R1-Distill Qwen-14B, Feature Category=Lexical Baselines2026.05 | 0.501 | |
| TF-IDF last sentenceModel=R1-Distill Qwen-14B, Feature Category=Lexical Baselines2026.05 | 0.489 | |
| TF-IDF prefixModel=GPT-OSS-20B, Feature Category=Lexical Baselines2026.05 | 0.482 | |
| TF-IDF prefixModel=R1-Distill Llama-8B, Feature Category=Lexical Baselines2026.05 | 0.466 |