Question Answering on LongBench-SingleDoc (test)
26.8Score (2k tokens)LLMLingua-2
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LLMLingua-2Target LLM=Mistral-7B, Constraint Level=2,000-token, Tokens=1,967, Speedup (1/τ)=7.4x2024.03 | 26.8 | — | |
| LLMLingua-2-smallTarget LLM=Mistral-7B, Constraint Level=2,000-token, Tokens=1,949, Speedup (1/τ)=7.4x2024.03 | 25.3 | — | |
| Original PromptTarget LLM=Mistral-7B, Tokens=14,5112024.03 | 24.5 | 24.5 | |
| Selective-ContextTarget LLM=Mistral-7B, Constraint Level=2,000-token, Tokens=2,038, Speedup (1/τ)=7.1x2024.03 | 22 | — | |
| LLMLinguaTarget LLM=Mistral-7B, Constraint Level=2,000-token, Tokens=2,054, Speedup (1/τ)=7.1x2024.03 | 19.5 | — | |
| LLMLinguaTarget LLM=Mistral-7B, Constraint Level=3,000-token, Tokens=3,076, Speedup (1/τ)=4.7x2024.03 | — | 20.8 | |
| LLMLingua-2Target LLM=Mistral-7B, Constraint Level=3,000-token, Tokens=2,853, Speedup (1/τ)=5.1x2024.03 | — | 27.3 | |
| LLMLingua-2-smallTarget LLM=Mistral-7B, Constraint Level=3,000-token, Tokens=2,888, Speedup (1/τ)=5.0x2024.03 | — | 27.9 | |
| Selective-ContextTarget LLM=Mistral-7B, Constraint Level=3,000-token, Tokens=3,075, Speedup (1/τ)=4.7x2024.03 | — | 26 |