Paraphrase Detection on MRPC (Accuracy Statistics)
89.9Avg AccuracyCAME
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CAMEmode=fine-tuning, batch size=8k2023.07 | 89.9 | — | — | |
| CAMEmode=fine-tuning, batch size=32k2023.07 | 89.8 | — | — | |
| ColD-Fusion2022.12 | 89.26 | — | — | |
| MUPPET2022.12 | 88.97 | — | — | |
| Baselinemode=fine-tuning2023.07 | 88.9 | — | — | |
| Finetune2022.12 | 87.45 | — | — | |
| RLLRBackbone=Mistral 7B2024.05 | 87.01 | — | — | |
| Multitask2022.12 | 86.96 | — | — | |
| RLLR_MIXEDBackbone=Mistral 7B2024.05 | 86.76 | — | — | |
| SFT w. rat.Backbone=Mistral 7B2024.05 | 86.52 | — | — | |
| RLHFModel=LLaMA2, Model Size=13B2024.05 | 85.78 | — | — | |
| RLLRModel=LLaMA2, Model Size=13B2024.05 | 85.54 | — | — | |
| RLLRMIXEDModel=LLaMA2, Model Size=13B2024.05 | 85.54 | — | — | |
| RLHFBackbone=Mistral 7B2024.05 | 85.29 | — | — | |
| RLLR_MIXEDBackbone=Bloom 7B2024.05 | 84.31 | — | — | |
| RLLRMIXEDModel=Bloom, Model Size=7B2024.05 | 84.31 | — | — | |
| RLLR_MIXEDBackbone=LLaMA2 7B2024.05 | 84.07 | — | — | |
| RLLRBackbone=Bloom 7B2024.05 | 84.07 | — | — | |
| RLLRMIXEDModel=LLaMA2, Model Size=7B2024.05 | 84.07 | — | — | |
| RLLRModel=Bloom, Model Size=7B2024.05 | 84.07 | — | — | |
| RLHFBackbone=LLaMA2 7B2024.05 | 83.82 | — | — | |
| RLLRBackbone=LLaMA2 7B2024.05 | 83.82 | — | — | |
| RLHFModel=LLaMA2, Model Size=7B2024.05 | 83.82 | — | — | |
| RLLRModel=LLaMA2, Model Size=7B2024.05 | 83.82 | — | — | |
| SFT w. rat.Model=LLaMA2, Model Size=13B2024.05 | 83.82 | — | — | |
| SFT w. rat.Backbone=LLaMA2 7B2024.05 | 83.58 | — | — | |
| SFTBackbone=Mistral 7B2024.05 | 83.58 | — | — | |
| RLLR_MIXEDBackbone=Baichuan2 7B2024.05 | 83.58 | — | — | |
| SFT w. rat.Model=LLaMA2, Model Size=7B2024.05 | 83.58 | — | — | |
| RLHFBackbone=Baichuan2 7B2024.05 | 83.09 | — | — | |
| RLLRBackbone=Baichuan2 7B2024.05 | 82.84 | — | — | |
| SFT w. rat.Backbone=Bloom 7B2024.05 | 82.6 | — | — | |
| SFT w. rat.Model=Bloom, Model Size=7B2024.05 | 82.6 | — | — | |
| RLLR_MIXEDBackbone=ChatGLM3 6B2024.05 | 82.36 | — | — | |
| RLHFBackbone=ChatGLM3 6B2024.05 | 82.35 | — | — | |
| RLLRBackbone=ChatGLM3 6B2024.05 | 82.35 | — | — | |
| SFT w. rat.Backbone=Baichuan2 7B2024.05 | 82.35 | — | — | |
| SFTBackbone=LLaMA2 7B2024.05 | 82.11 | — | — | |
| RLHFBackbone=Bloom 7B2024.05 | 82.11 | — | — | |
| SFTModel=LLaMA2, Model Size=7B2024.05 | 82.11 | — | — | |
| RLHFModel=Bloom, Model Size=7B2024.05 | 82.11 | — | — | |
| SFT w. rat.Backbone=ChatGLM3 6B2024.05 | 81.86 | — | — | |
| SFTBackbone=Baichuan2 7B2024.05 | 81.86 | — | — | |
| SFTModel=LLaMA2, Model Size=13B2024.05 | 81.62 | — | — | |
| SFTBackbone=ChatGLM3 6B2024.05 | 81.37 | — | — | |
| RLLRModel=Bloom, Model Size=3B2024.05 | 80.64 | — | — | |
| RLLRMIXEDModel=Bloom, Model Size=3B2024.05 | 80.64 | — | — | |
| SFT w. rat.Model=Bloom, Model Size=3B2024.05 | 80.15 | — | — | |
| RLHFModel=Bloom, Model Size=3B2024.05 | 79.66 | — | — | |
| SFTBackbone=Bloom 7B2024.05 | 76.96 | — | — | |
| SFTModel=Bloom, Model Size=7B2024.05 | 76.96 | — | — | |
| SFTModel=Bloom, Model Size=3B2024.05 | 75.25 | — | — | |
| FINETUNE2022.04 | 70.1 | 62.7 | 4.7 | |
| LLaMA2-7BEvaluation protocol=Zero-shot, Parameters=7B2024.02 | 69.1 | — | — | |
| LLaMA-7BEvaluation protocol=Zero-shot, Parameters=7B2024.02 | 68.6 | — | — | |
| bitfit+mteMode=Ablation2022.04 | 68.5 | 57.4 | 5.1 | |
| OLMo-7BEvaluation protocol=Zero-shot, Parameters=7B2024.02 | 68.4 | — | — | |
| PERFECTInitialization=random2022.04 | 67.8 | 54.7 | 5.7 | |
| MPT-7BEvaluation protocol=Zero-shot, Parameters=7B2024.02 | 67.7 | — | — | |
| FastGASAnnotation budget (|L|)=100, Prompt retrieval=similarity-based2024.06 | 66.15 | — | — | |
| FastGASModel=GPT-Neo-2.7B, Annotation Budget=182024.06 | 66.02 | — | — | |
| PERFECTInitialization=prototype-based2022.04 | 65.9 | 56.6 | 6 | |
| IDEALModel=GPT-Neo-2.7B, Annotation Budget=182024.06 | 65.89 | — | — | |
| IDEALAnnotation budget (|L|)=100, Prompt retrieval=similarity-based2024.06 | 65.49 | — | — | |
| Pythia-6.9BEvaluation protocol=Zero-shot, Parameters=6.9B2024.02 | 65.4 | — | — | |
| FastGASAnnotation budget (|L|)=18, Prompt retrieval=similarity-based2024.06 | 65.23 | — | — | |
| PageRankAnnotation budget (|L|)=100, Prompt retrieval=similarity-based2024.06 | 64.84 | — | — | |
| SubclusteringAnnotation budget (|L|)=100, Prompt retrieval=similarity-based2024.06 | 64.84 | — | — | |
| Vote-kAnnotation budget (|L|)=100, Prompt retrieval=similarity-based2024.06 | 64.6 | — | — | |
| RandomAnnotation budget (|L|)=100, Prompt retrieval=similarity-based2024.06 | 64.32 | — | — | |
| Logan IV et al. (2021)2022.04 | 63.9 | 53.7 | 5.3 | |
| IDEALAnnotation budget (|L|)=18, Prompt retrieval=similarity-based2024.06 | 63.8 | — | — | |
| PETStrategy=Best2022.04 | 63.4 | 49.3 | 6.5 | |
| Falcon-7BEvaluation protocol=Zero-shot, Parameters=7B2024.02 | 62.8 | — | — | |
| Top-degreeAnnotation budget (|L|)=100, Prompt retrieval=similarity-based2024.06 | 62.76 | — | — | |
| PETStrategy=Average2022.04 | 62.1 | 38.2 | 6.8 | |
| SubclusteringAnnotation budget (|L|)=18, Prompt retrieval=similarity-based2024.06 | 61.46 | — | — | |
| LouvainAnnotation budget (|L|)=18, Prompt retrieval=similarity-based2024.06 | 61.33 | — | — | |
| prompt+mteMode=Ablation2022.04 | 60 | 51.5 | 5.8 | |
| LouvainAnnotation budget (|L|)=100, Prompt retrieval=similarity-based2024.06 | 59.63 | — | — | |
| RPJ-INCITE-7BEvaluation protocol=Zero-shot, Parameters=7B2024.02 | 58.8 | — | — | |
| Vote-kModel=GPT-Neo-2.7B, Annotation Budget=182024.06 | 57.42 | — | — | |
| Vote-kAnnotation budget (|L|)=18, Prompt retrieval=similarity-based2024.06 | 56.9 | — | — | |
| RandomAnnotation budget (|L|)=18, Prompt retrieval=similarity-based2024.06 | 55.47 | — | — | |
| Top-degreeAnnotation budget (|L|)=18, Prompt retrieval=similarity-based2024.06 | 48.57 | — | — | |
| PageRankAnnotation budget (|L|)=18, Prompt retrieval=similarity-based2024.06 | 46.09 | — | — | |
| IDEALModel=OPT-6.7B, Annotation Budget=182024.06 | 45.18 | — | — | |
| FastGASModel=OPT-6.7B, Annotation Budget=182024.06 | 44.66 | — | — | |
| Vote-kModel=OPT-6.7B, Annotation Budget=182024.06 | 36.59 | — | — |