Question Answering on Question Answering (ASR)
80ASR AccuracyCOMA
Evaluation Results
| Method | Links | |
|---|---|---|
| COMACompressor=SC, Compressor Category=Extractive2025.10 | 80 | |
| COMACompressor=LLMLingua2, Compressor Category=Extractive2025.10 | 72 | |
| COMACompressor=Gemma-3-4B, Compressor Category=Abstractive2025.10 | 71 | |
| COMACompressor=Llama-3.2-3B, Compressor Category=Abstractive2025.10 | 69 | |
| COMACompressor=LLMLingua, Compressor Category=Extractive2025.10 | 62 | |
| COMACompressor=Qwen3-4B, Compressor Category=Abstractive2025.10 | 59 | |
| JudgeDeceiverCompressor=Gemma-3-4B, Compressor Category=Abstractive2025.10 | 36 | |
| JudgeDeceiverCompressor=Llama-3.2-3B, Compressor Category=Abstractive2025.10 | 32 | |
| JudgeDeceiverCompressor=Qwen3-4B, Compressor Category=Abstractive2025.10 | 32 | |
| Escape CharactersCompressor=SC, Compressor Category=Extractive2025.10 | 24 | |
| Context IgnoreCompressor=LLMLingua2, Compressor Category=Extractive2025.10 | 24 | |
| Direct InjectionCompressor=SC, Compressor Category=Extractive2025.10 | 22 | |
| Context IgnoreCompressor=SC, Compressor Category=Extractive2025.10 | 21 | |
| Naive AttackCompressor=SC, Compressor Category=Extractive2025.10 | 19 | |
| Escape CharactersCompressor=LLMLingua, Compressor Category=Extractive2025.10 | 19 | |
| Context IgnoreCompressor=LLMLingua, Compressor Category=Extractive2025.10 | 18 | |
| Escape CharactersCompressor=LLMLingua2, Compressor Category=Extractive2025.10 | 18 | |
| JudgeDeceiverCompressor=SC, Compressor Category=Extractive2025.10 | 17 | |
| Direct InjectionCompressor=LLMLingua, Compressor Category=Extractive2025.10 | 16 | |
| Naive AttackCompressor=LLMLingua, Compressor Category=Extractive2025.10 | 15 | |
| JudgeDeceiverCompressor=LLMLingua, Compressor Category=Extractive2025.10 | 15 | |
| JudgeDeceiverCompressor=LLMLingua2, Compressor Category=Extractive2025.10 | 9 | |
| Direct InjectionCompressor=LLMLingua2, Compressor Category=Extractive2025.10 | 8 | |
| Context IgnoreCompressor=Llama-3.2-3B, Compressor Category=Abstractive2025.10 | 5 | |
| Naive AttackCompressor=LLMLingua2, Compressor Category=Extractive2025.10 | 4 | |
| Context IgnoreCompressor=Gemma-3-4B, Compressor Category=Abstractive2025.10 | 4 | |
| Context IgnoreCompressor=Qwen3-4B, Compressor Category=Abstractive2025.10 | 2 | |
| Escape CharactersCompressor=Gemma-3-4B, Compressor Category=Abstractive2025.10 | 1 | |
| No AttackCompressor=SC, Compressor Category=Extractive2025.10 | 0 | |
| No AttackCompressor=LLMLingua, Compressor Category=Extractive2025.10 | 0 | |
| No AttackCompressor=LLMLingua2, Compressor Category=Extractive2025.10 | 0 | |
| No AttackCompressor=Llama-3.2-3B, Compressor Category=Abstractive2025.10 | 0 | |
| Naive AttackCompressor=Llama-3.2-3B, Compressor Category=Abstractive2025.10 | 0 | |
| Direct InjectionCompressor=Llama-3.2-3B, Compressor Category=Abstractive2025.10 | 0 | |
| Escape CharactersCompressor=Llama-3.2-3B, Compressor Category=Abstractive2025.10 | 0 | |
| No AttackCompressor=Qwen3-4B, Compressor Category=Abstractive2025.10 | 0 | |
| Naive AttackCompressor=Qwen3-4B, Compressor Category=Abstractive2025.10 | 0 | |
| Direct InjectionCompressor=Qwen3-4B, Compressor Category=Abstractive2025.10 | 0 | |
| Escape CharactersCompressor=Qwen3-4B, Compressor Category=Abstractive2025.10 | 0 | |
| No AttackCompressor=Gemma-3-4B, Compressor Category=Abstractive2025.10 | 0 | |
| Naive AttackCompressor=Gemma-3-4B, Compressor Category=Abstractive2025.10 | 0 | |
| Direct InjectionCompressor=Gemma-3-4B, Compressor Category=Abstractive2025.10 | 0 | |
| COMACompressor=None, Compressor Category=None2025.10 | 0 |