Natural Language Understanding on GLUE (SST2, MRPC, RTE, MNLI Subset)
96.56SST2 AccuracyCentSFT
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CentSFTModel=Mistral-7B, Ratio=-2026.04 | 96.56 | 89.22 | 88.81 | 88.35 | 90.74 | |
| CentSFTModel=LLaMA2-7B, Ratio=-2026.04 | 96.33 | 85.05 | 86.28 | 86.77 | 88.61 | |
| FedProxyModel=Mistral-7B, Ratio=502026.04 | 96.1 | 80.39 | 85.92 | 80.91 | 85.83 | |
| FedProxyModel=Mistral-7B, Ratio=302026.04 | 95.64 | 83.82 | 85.92 | 84.08 | 87.37 | |
| FedProxyModel=LLaMA2-7B, Ratio=302026.04 | 95.41 | 81.13 | 82.67 | 80.56 | 84.94 | |
| FedProxyModel=LLaMA2-7B, Ratio=502026.04 | 95.41 | 77.21 | 84.84 | 60.29 | 79.44 | |
| OTModel=Mistral-7B, Ratio=302026.04 | 94.04 | 73.53 | 78.34 | 72.64 | 79.64 | |
| FedOTModel=Mistral-7B, Ratio=302026.04 | 93 | 73.77 | 78.34 | 70.66 | 78.94 | |
| FedBiOTModel=Mistral-7B, Ratio=502026.04 | 91.4 | 73.04 | 71.84 | 60.36 | 74.16 | |
| FedBiOTModel=Mistral-7B, Ratio=302026.04 | 91.28 | 73.04 | 74.01 | 62.58 | 75.23 | |
| OTModel=Mistral-7B, Ratio=502026.04 | 90.48 | 73.53 | 75.09 | 61.74 | 75.21 | |
| FedOTModel=Mistral-7B, Ratio=502026.04 | 90.02 | 71.81 | 71.48 | 61.01 | 73.58 | |
| ZeroShotModel=Mistral-7B, Ratio=-2026.04 | 86.7 | 73.04 | 72.2 | 59.52 | 72.87 | |
| FedBiOTModel=LLaMA2-7B, Ratio=302026.04 | 84.17 | 63.24 | 62.09 | 46.52 | 64.01 | |
| FedOTModel=LLaMA2-7B, Ratio=302026.04 | 82.8 | 68.87 | 69.31 | 41.07 | 65.51 | |
| FedOTModel=LLaMA2-7B, Ratio=502026.04 | 65.14 | 68.63 | 62.45 | 41.08 | 59.33 | |
| OTModel=LLaMA2-7B, Ratio=502026.04 | 53.9 | 69.36 | 62.45 | 36 | 55.43 | |
| FedBiOTModel=LLaMA2-7B, Ratio=502026.04 | 51.26 | 69.36 | 61.37 | 46.52 | 57.13 | |
| ZeroShotModel=LLaMA2-7B, Ratio=-2026.04 | 49.54 | 68.87 | 61.37 | 39.96 | 54.94 | |
| OTModel=LLaMA2-7B, Ratio=302026.04 | 49.43 | 70.1 | 57.04 | 49.38 | 56.49 |