Sentence-pair classification on QQP
88.8AccuracyRLLR
Evaluation results
| Method | Links | ||
|---|---|---|---|
| RLLRModel=LLaMA2, Model Size=13B2024.05 | 88.8 | — | |
| RLLRMIXEDModel=LLaMA2, Model Size=13B2024.05 | 88.8 | — | |
| RLHFModel=LLaMA2, Model Size=13B2024.05 | 88.7 | — | |
| SFT w. rat.Model=LLaMA2, Model Size=13B2024.05 | 88.4 | — | |
| RLLRModel=LLaMA2, Model Size=7B2024.05 | 88.2 | — | |
| RLLRMIXEDModel=LLaMA2, Model Size=7B2024.05 | 88 | — | |
| SFTModel=LLaMA2, Model Size=13B2024.05 | 88 | — | |
| SFT w. rat.Model=LLaMA2, Model Size=7B2024.05 | 87.9 | — | |
| RLHFModel=LLaMA2, Model Size=7B2024.05 | 87.6 | — | |
| SFTModel=LLaMA2, Model Size=7B2024.05 | 85.5 | — | |
| RLLRModel=Bloom, Model Size=7B2024.05 | 84.3 | — | |
| RLLRMIXEDModel=Bloom, Model Size=7B2024.05 | 84.3 | — | |
| RLHFModel=Bloom, Model Size=7B2024.05 | 84 | — | |
| SFT w. rat.Model=Bloom, Model Size=7B2024.05 | 83.6 | — | |
| SFTModel=Bloom, Model Size=7B2024.05 | 83.4 | — | |
| RLLRModel=Bloom, Model Size=3B2024.05 | 82.6 | — | |
| RLLRMIXEDModel=Bloom, Model Size=3B2024.05 | 82.4 | — | |
| RLHFModel=Bloom, Model Size=3B2024.05 | 82.2 | — | |
| SFT w. rat.Model=Bloom, Model Size=3B2024.05 | 81.9 | — | |
| SFTModel=Bloom, Model Size=3B2024.05 | 81.3 | — | |
| Llama-3-8BBackbone=Llama-3-8B2025.05 | 79.7 | — | |
| ASTRABackbone=Llama-3-8B, Group Size (G)=322025.05 | 78.03 | — | |
| ASTRABackbone=Llama-3-8B, Group Size (G)=12025.05 | 77.03 | — | |
| ASTRABackbone=Llama-3-8B, Group Size (G)=162025.05 | 76.74 | — | |
| BERT (Full Clean Training)Training Data=Full clean dataset2022.10 | 0.9141 | — | |
| Backdoored Model (No Defense)Backdoor Attack=Word+EP, Clean training instances=642022.10 | 0.9138 | 99.98 | |
| Backdoored Model (No Defense)Backdoor Attack=Trigger Sentence, Clean training instances=642022.10 | 0.9097 | 100 | |
| Backdoored Model (No Defense)Backdoor Attack=Trigger Word, Clean training instances=642022.10 | 0.9089 | 100 | |
| Backdoored Model (No Defense)Backdoor Attack=Word (Scratch), Clean training instances=642022.10 | 0.8971 | 100 | |
| Fine-pruningBackdoor Attack=Trigger Word, Clean training instances=642022.10 | 0.8564 | 100 | |
| Fine-mixingBackdoor Attack=Trigger Word, Clean training instances=642022.10 | 0.85 | 56.87 | |
| Fine-pruningBackdoor Attack=Word (Scratch), Clean training instances=642022.10 | 0.8458 | 100 | |
| Fine-mixingBackdoor Attack=Word (Scratch), Clean training instances=642022.10 | 0.8319 | 69.39 | |
| Fine-mixingBackdoor Attack=Sentence (Scratch), Clean training instances=642022.10 | 0.8237 | 88.71 | |
| Fine-mixingBackdoor Attack=Word+EP, Clean training instances=642022.10 | 0.8232 | 15.4 | |
| Fine-mixingBackdoor Attack=Word (Scratch), Clean training instances=1282022.10 | 0.8125 | 38.55 | |
| Fine-mixingBackdoor Attack=Sentence (Scratch), Clean training instances=2562022.10 | 0.8106 | 41.14 | |
| Fine-mixingBackdoor Attack=Trigger Sentence, Clean training instances=642022.10 | 0.8093 | 42.66 | |
| Fine-mixingBackdoor Attack=Sentence (Scratch), Clean training instances=1282022.10 | 0.8058 | 46.31 | |
| Fine-mixingBackdoor Attack=Sentence (Scratch), Clean training instances=5122022.10 | 0.8033 | 37.75 | |
| Initial BERT (Clean fine-tuned)Clean training instances=5122022.10 | 0.772 | — | |
| Initial BERT (Clean fine-tuned)Clean training instances=2562022.10 | 0.7337 | — | |
| Initial BERT (Clean fine-tuned)Clean training instances=1282022.10 | 0.6978 | — | |
| Initial BERT (Clean fine-tuned)Clean training instances=642022.10 | 0.6495 | — |