List your company
ResearchFormerly WizWand

System Prompt Corruption (SPC) on COMA Evaluation Suite

100DSRIsolated Compression (IC)
-0.8825.3151.577.69Oct 27, 2025
Updated 4mo ago

Evaluation results

MethodLinks
Isolated Compression (IC)Compressor=LLMLingua22025.10100
Isolated Compression (IC)Compressor=SC2025.1099
Isolated Compression (IC)Compressor=LLMLingua2025.1099
Isolated Compression (IC)Compressor=Qwen3-4B2025.1096
Isolated Compression (IC)Compressor=Llama-3.2-3B2025.1092
Isolated Compression (IC)Compressor=Gemma-3-4B2025.1092
Window-PPL (W-PPL)Compressor=LLMLingua2025.1022
PPLCompressor=LLMLingua2025.1018
Window-PPL (W-PPL)Compressor=SC2025.1017
Window-PPL (W-PPL)Compressor=Qwen3-4B2025.1016
StruQCompressor=Qwen3-4B2025.1016
Window-PPL (W-PPL)Compressor=Gemma-3-4B2025.1015
Window-PPL (W-PPL)Compressor=Llama-3.2-3B2025.1013
StruQCompressor=LLMLingua22025.1012
PPLCompressor=Qwen3-4B2025.1011
SecAlignCompressor=SC2025.1011
PPLCompressor=SC2025.109
PPLCompressor=Llama-3.2-3B2025.109
PPLCompressor=Gemma-3-4B2025.109
Window-PPL (W-PPL)Compressor=LLMLingua22025.109
StruQCompressor=LLMLingua2025.109
StruQCompressor=Gemma-3-4B2025.109
StruQCompressor=SC2025.108
StruQCompressor=Llama-3.2-3B2025.108
SecAlignCompressor=LLMLingua22025.108
SecAlignCompressor=Qwen3-4B2025.108
SecAlignCompressor=LLMLingua2025.106
SecAlignCompressor=Llama-3.2-3B2025.105
PPLCompressor=LLMLingua22025.104
SecAlignCompressor=Gemma-3-4B2025.103