Question Answering on NQ (F1, EM)
75.96F1 ScorePCC-Large
Evaluation results
| Method | Links | ||
|---|---|---|---|
| PCC-LargeCompression Rate=4x, Parameter Size=8B2026.02 | 75.96 | 67.71 | |
| OURS (PIC)Compression Rate=4x, Parameter Size=0.5B2026.02 | 71.82 | 62.47 | |
| PCC-LargeCompression Rate=16x, Parameter Size=8B2026.02 | 71.09 | 61.97 | |
| PCC-LiteCompression Rate=4x, Parameter Size=0.77B2026.02 | 70.51 | 61.97 | |
| LLMLingua2Compression Rate=4x, Parameter Size=0.6B2026.02 | 68.44 | 55.85 | |
| OURS (PIC)Compression Rate=16x, Parameter Size=1B2026.02 | 68.17 | 58.9 | |
| OURS (PIC)Compression Rate=16x, Parameter Size=0.5B2026.02 | 67.61 | 58.37 | |
| w/ ContextCompression Rate=1x2026.02 | 64.64 | 51.38 | |
| PCC-LiteCompression Rate=16x, Parameter Size=0.77B2026.02 | 62.56 | 52.73 | |
| ICAECompression Rate=4x, Parameter Size=7B2026.02 | 59.15 | 47.35 | |
| OURS (PIC)Compression Rate=64x, Parameter Size=0.5B2026.02 | 57.96 | 49.15 | |
| COCOM-LiteCompression Rate=4x, Parameter Size=0.1B2026.02 | 50.45 | 41.87 | |
| xRAGCompression Rate=> 64x2026.02 | 38.06 | 20.8 | |
| PCC-LiteCompression Rate=64x, Parameter Size=0.77B2026.02 | 36.82 | 29.18 | |
| w/o ContextCompression Rate=1x2026.02 | 27.48 | 15.02 | |
| COCOM-LiteCompression Rate=16x, Parameter Size=0.1B2026.02 | 26.36 | 20.66 | |
| AutoCompressorCompression Rate=40x2026.02 | 25.57 | 0.63 |