Summarization on BillSum
69.6AccuracyDense Model
Evaluation Results
| Method | Links | |
|---|---|---|
| Dense ModelBase Model=Llama3.1-8B, Fine-tuning Protocol=FFT, Sparsity=0%2026.01 | 69.6 | |
| FT(Llama3.2)Base Model=Llama3.2-3B, Fine-tuning Protocol=FFT, Sparsity=0%2026.01 | 68.8 | |
| GradPrunerBase Model=Llama3.1-8B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 68.7 | |
| Dense ModelBase Model=Mistral-7B, Fine-tuning Protocol=FFT, Sparsity=0%2026.01 | 68.4 | |
| Dense ModelBase Model=Mistral-7B, Fine-tuning Protocol=LoRA, Sparsity=0%2026.01 | 68.1 | |
| Dense ModelBase Model=Llama3.1-8B, Fine-tuning Protocol=LoRA, Sparsity=0%2026.01 | 67.7 | |
| GradPrunerBase Model=Mistral-7B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 67 | |
| FT(Llama3.2)Base Model=Llama3.2-3B, Fine-tuning Protocol=LoRA, Sparsity=0%2026.01 | 66.2 | |
| GradPrunerBase Model=Llama3.1-8B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 65.9 | |
| GradPrunerBase Model=Mistral-7B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 65.9 | |
| SATBase Model=Mistral-7B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 65.7 | |
| SATBase Model=Llama3.1-8B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 65.6 | |
| MINITRONBase Model=Mistral-7B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 65 | |
| LacoBase Model=Llama3.1-8B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 64.9 | |
| APTBase Model=Llama3.1-8B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 64.5 | |
| LacoBase Model=Mistral-7B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 64.3 | |
| LLMPrunerBase Model=Llama3.1-8B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 64.1 | |
| LacoBase Model=Mistral-7B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 64.1 | |
| MINITRONBase Model=Llama3.1-8B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 64 | |
| LLMPrunerBase Model=Mistral-7B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 63.9 | |
| LLMPrunerBase Model=Llama3.1-8B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 63.6 | |
| MINITRONBase Model=Mistral-7B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 63.3 | |
| LacoBase Model=Llama3.1-8B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 63.1 | |
| MINITRONBase Model=Llama3.1-8B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 63.1 | |
| APTBase Model=Mistral-7B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 63.1 | |
| SATBase Model=Mistral-7B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 62.9 | |
| LLMPrunerBase Model=Mistral-7B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 62.5 | |
| SATBase Model=Llama3.1-8B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 62.1 |