List your company
ResearchFormerly WizWand

Idioms on MAGPIE

98.8AUROCFull Rep.
93.39294.79696.297.604Apr 20, 2026
Updated 6mo ago

Evaluation results

MethodLinks
Full Rep.Model=GPT-OSS-20B, Representation Type=Full Representation2026.0498.8
Full Rep.Model=Gemma2-9B, Representation Type=Full Representation2026.0498.7
Full Representation ClassifierBackbone=Llama-3.1-8B, Representation Type=Full Rep., Evaluation Protocol=Trained2026.0498.5
SubspaceModel=Gemma2-9B, Representation Type=Unidirectional Concreteness Subspace2026.0497.4
Full Rep.Model=Qwen3-8B, Representation Type=Full Representation2026.0497.2
one-directional concreteness axisBackbone=Llama-3.1-8B, Representation Type=Subspace, Evaluation Protocol=Zero-shot2026.0495.2
SubspaceModel=Qwen3-8B, Representation Type=Unidirectional Concreteness Subspace2026.0494.3
SubspaceModel=GPT-OSS-20B, Representation Type=Unidirectional Concreteness Subspace2026.0493.6