Text Compression on Reddit conversational text n=200 (test)
99.46BERTScore F1 (r_keep=0.9)Llama SFT
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Llama SFTBase model=Llama-3.2-3B-Instruct, Evaluation Protocol=QLoRA fine-tuning2026.05 | 99.46 | 99.27 | 95.9 | 90.19 | 85.25 | |
| WF-3class (char.)targeting_level=character-level2026.05 | 99.37 | 99.24 | 93.92 | 85.73 | 81.84 | |
| Hybrid-0.3alpha=0.3, pipeline=GPT-2 surprisal + Gemini 2.0 Flash2026.05 | 99.34 | 97.83 | 94.25 | 89.69 | 85.05 | |
| Hybrid-0.5alpha=0.5, pipeline=GPT-2 surprisal + Gemini 2.0 Flash2026.05 | 99.34 | 97.82 | 93.93 | 89.51 | 84.91 | |
| Hybrid-0.7alpha=0.7, pipeline=GPT-2 surprisal + Gemini 2.0 Flash2026.05 | 99.31 | 97.96 | 93.63 | 88.49 | 84.65 | |
| Steptargeting_level=token-level2026.05 | 99.28 | 96.32 | 84.65 | 82.84 | 81.61 | |
| WordLentargeting_level=token-level2026.05 | 99.26 | 99.13 | 93.78 | 85.72 | 81.94 | |
| Entropypipeline=GPT-2 surprisal + Gemini 2.0 Flash2026.05 | 99.24 | 97.62 | 94.29 | 89.36 | 85.03 | |
| Entropy-LPpipeline=GPT-2 surprisal + Gemini 2.0 Flash2026.05 | 99.24 | 97.81 | 94.32 | 89.61 | 85.12 | |
| WF-Opttargeting_level=token-level2026.05 | 99.11 | 97.66 | 93.7 | 88.48 | 83.75 | |
| Entropy-in-FreqBucketspipeline=GPT-2 surprisal + Gemini 2.0 Flash2026.05 | 98.87 | 95.68 | 92.12 | 88.44 | 84.95 | |
| WF-6classtargeting_level=token-level2026.05 | 98.54 | 95.68 | 91.41 | 86.65 | 83.46 | |
| WF-3classtargeting_level=token-level2026.05 | 98.36 | 95.91 | 92.31 | 88.41 | 84.39 | |
| Llama baseBase model=Llama-3.2-3B-Instruct, Evaluation Protocol=zero-shot2026.05 | 97.8 | 95.65 | 85.05 | 86.68 | 83.71 |