Idioms on MAGPIE
98.8AUROCFull Rep.
Evaluation Results
| Method | Links | |
|---|---|---|
| Full Rep.Model=GPT-OSS-20B, Representation Type=Full Representation2026.04 | 98.8 | |
| Full Rep.Model=Gemma2-9B, Representation Type=Full Representation2026.04 | 98.7 | |
| Full Representation ClassifierBackbone=Llama-3.1-8B, Representation Type=Full Rep., Evaluation Protocol=Trained2026.04 | 98.5 | |
| SubspaceModel=Gemma2-9B, Representation Type=Unidirectional Concreteness Subspace2026.04 | 97.4 | |
| Full Rep.Model=Qwen3-8B, Representation Type=Full Representation2026.04 | 97.2 | |
| one-directional concreteness axisBackbone=Llama-3.1-8B, Representation Type=Subspace, Evaluation Protocol=Zero-shot2026.04 | 95.2 | |
| SubspaceModel=Qwen3-8B, Representation Type=Unidirectional Concreteness Subspace2026.04 | 94.3 | |
| SubspaceModel=GPT-OSS-20B, Representation Type=Unidirectional Concreteness Subspace2026.04 | 93.6 |