Text Classification on AGNews (Accuracy and Speedup)
95.68AccuracyLS-unLLaMA-2-7B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LS-unLLaMA-2-7BModel Size=7B2023.10 | 95.68 | — | |
| LS-LLaMA-2-13BModel Size=13B2023.10 | 95.66 | — | |
| LS-unLLaMA-2-13BModel Size=13B2023.10 | 95.44 | — | |
| LS-LLaMA-2-7BModel Size=7B2023.10 | 95.38 | — | |
| RoBERTa-LargeModel Architecture=Discriminant baseline2023.10 | 94.78 | — | |
| RoBERTa-BaseModel Architecture=Discriminant baseline2023.10 | 94.7 | — | |
| BERT-BaseModel Architecture=Discriminant baseline2023.10 | 94.51 | — | |
| BERT-LargeModel Architecture=Discriminant baseline2023.10 | 94.45 | — | |
| AGNModel Architecture=Discriminant baseline2023.10 | 93.82 | — | |
| DepGraphArchitecture=LSTM2023.01 | 91.75 | 16.28 | |
| DepGraph w/o SLArchitecture=LSTM2023.01 | 91.53 | 16.28 | |
| DepGraph+CPArchitecture=LSTM2023.01 | 91.5 | 16.28 | |
| DepGraph+RandomArchitecture=LSTM2023.01 | 91.23 | 16.28 | |
| SCModel=Mistral, k-shot=8, Seeds=52025.05 | 87.42 | — | |
| DCModel=Qwen, k-shot=8, Seeds=52025.05 | 87.27 | — | |
| SCModel=Llama, k-shot=8, Seeds=52025.05 | 86.56 | — | |
| SCModel=Qwen, k-shot=8, Seeds=52025.05 | 86.02 | — | |
| GPT-3 175BEvaluation Protocol=few-shot, Model Size=175B2023.10 | 84.3 | — | |
| CCModel=Qwen, k-shot=8, Seeds=52025.05 | 83.98 | — | |
| T5 3B# Params=8.5x, Evaluation Protocol=Zero-shot2022.12 | 80.5 | — | |
| Base LLMModel=Llama, k-shot=8, Seeds=52025.05 | 80.08 | — | |
| GPT-2 kNN-LM# Params=2.2x, Evaluation Protocol=Zero-shot2022.12 | 78.8 | — | |
| Base LLMModel=Mistral, k-shot=8, Seeds=52025.05 | 76.88 | — | |
| DCModel=Llama, k-shot=8, Seeds=52025.05 | 76.8 | — | |
| CCModel=Llama, k-shot=8, Seeds=52025.05 | 76.09 | — | |
| BCModel=Llama, k-shot=8, Seeds=52025.05 | 75.78 | — | |
| GPT-3# Params=500x, Evaluation Protocol=Zero-shot2022.12 | 75.4 | — | |
| GPT-3 + PMI# Params=500x, Evaluation Protocol=Zero-shot2022.12 | 74.7 | — | |
| BCModel=Qwen, k-shot=8, Seeds=52025.05 | 74.65 | — | |
| NPM# Params=1.0x, Evaluation Protocol=Zero-shot, nonparametric=true2022.12 | 74.5 | — | |
| CCModel=Mistral, k-shot=8, Seeds=52025.05 | 74.45 | — | |
| DCModel=Mistral, k-shot=8, Seeds=52025.05 | 74.3 | — | |
| BCModel=Mistral, k-shot=8, Seeds=52025.05 | 73.67 | — | |
| Base LLMModel=Qwen, k-shot=8, Seeds=52025.05 | 73.16 | — | |
| T5# Params=2.2x, Evaluation Protocol=Zero-shot2022.12 | 72 | — | |
| RoBERTa# Params=1.0x, Evaluation Protocol=Zero-shot2022.12 | 71.3 | — | |
| GPT-2# Params=2.2x, Evaluation Protocol=Zero-shot2022.12 | 67.4 | — | |
| GPT-2 + PMI# Params=2.2x, Evaluation Protocol=Zero-shot2022.12 | 65.1 | — | |
| LLaMA-2-13BEvaluation Protocol=zero-shot, Model Size=13B2023.10 | 59.4 | — | |
| LLaMA-2-7BEvaluation Protocol=instruction-tuning, Model Size=7B2023.10 | 52.4 | — | |
| GPT-3 175BEvaluation Protocol=zero-shot, Model Size=175B2023.10 | 43.9 | — | |
| LLaMA-2-7BEvaluation Protocol=zero-shot, Model Size=7B2023.10 | 37.39 | — | |
| GPT-2 kNN# Params=2.2x, Evaluation Protocol=Zero-shot2022.12 | 29.8 | — |