Natural Language Understanding on GLUE (CoLA, STS-B, MRPC, RTE, QQP, QNLI, SST-2, MNLI)

89.5Average Score (GLUE)PiSSA

Evaluation Results

Method
PiSSA2026.05	89.5	—	—	92.3	94.3	95.9	72.6	91.9	91.7	87	—	90.4	—	—
CERSA2026.05	89.5	—	—	92.4	94.4	96	72.3	91.7	92	87	—	90.3	—	—
SVFT2026.05	89	—	—	91.5	94.3	95.4	72.6	91.8	89	87.2	—	90	—	—
SVFit2026.05	88.6	—	—	90.2	94.3	95.4	71	91.8	88.8	87.4	—	89.7	—	—
LoRA2026.05	88.5	—	—	92	93.9	95	69.8	91.6	90	85.2	—	90.7	—	—
FT2026.05	88.3	—	—	92.4	94	95.6	69.2	91.6	89.5	83.8	—	89.9	—	—
GenFT2025.05	85.87	—	—	90.59	92.49	94.84	63.38	90.69	89.95	77.62	—	87.38	—	—
RoBERTa2026.06	85.5	—	—	91.7	92.7	94.8	61.9	90.2	88.5	76.9	—	87.6	—	—
PiSSA2025.05	85.42	—	—	88.37	91.8	93.92	66.03	90.4	88.73	78.7	—	85.41	—	—
GaLore2025.05	85.23	—	—	91.06	92.24	94.04	60.35	90.73	87.01	79.42	—	87	—	—
Propulsion2025.05	84.95	—	—	89.11	92.79	93.18	61.76	91.37	89.34	75.66	—	86.41	—	—
CorDA2025.05	84.83	—	—	91.1	91.49	93.12	59.6	90.17	89.71	76.17	—	87.28	—	—
LISA2025.05	84.55	—	—	88.37	90.72	94.04	62.29	89.72	89.22	77.26	—	84.79	—	—
FiRA2025.05	84.5	—	—	90.3	91.69	93.58	59.83	90.33	88.97	75.09	—	86.25	—	—
PROPETLv12025.05	84.46	—	—	89.22	92.56	92.18	63.11	91.33	85.25	75.54	—	86.49	—	—
AdapterS2025.05	84.31	—	—	90.81	90.84	92.14	60.32	90.25	89.24	73.56	—	87.33	—	—
BitFit2025.05	84.2	—	—	88.12	91.09	92.67	61.38	90.34	88.22	75.58	—	84.64	—	—
LoRA2025.05	83.99	—	—	88.83	92.02	92.4	60.09	90.66	88.5	72.92	—	86.54	—	—
Full Precision2026.04	83.9	84.9	85.5	91.4	88.6	93.2	59.7	89.8	86.2	72.2	—	—	—	—
Flexformer_n2026.06	83.7	—	—	91.1	91	93.6	63.3	88.3	85.8	71.5	—	85.1	—	—
MAMAdapter2025.05	83.67	—	—	88.31	90.19	93.19	58.42	90.74	89.31	72.62	—	86.63	—	—
Full FT2025.05	83.56	—	—	90.18	91.17	92.89	59.84	90.48	85.24	72.43	—	86.27	—	—
AdaLoRA2025.05	83.45	—	—	88.58	91.43	91.69	59.82	90.83	88.99	70.04	—	86.26	—	—
PROPETLv32025.05	83.45	—	—	88.9	92.06	92.54	61.72	90.76	87.42	67.39	—	86.84	—	—
Hedgehog2026.06	83.3	—	—	91.2	91.2	93.3	62.5	87.7	86	71	—	83.7	—	—
ModernBERT-base2026.06	83.01	—	—	—	—	—	—	—	—	—	—	—	—	—
(IA)32025.05	82.85	—	—	87.99	90.88	92.02	59.58	90.3	87	71.12	—	83.95	—	—
BEBERT2026.04	82.4	84.7	84.7	91.4	91.5	93.4	56.5	—	85.2	71.5	—	—	—	—
EvoGM2026.05	82.4	—	—	84.8	88.1	90	71.1	80.9	78.9	82.2	—	82.8	—	—
Full Precision2026.04	82.3	84.9	85.5	91.4	92.1	93.2	59.7	90.1	86.3	72.2	—	—	—	—
PROPETLv22025.05	82.26	—	—	88.54	91.51	91.36	60.18	90.3	86.73	63.31	—	86.22	—	—
PSO-Merging2026.05	81.2	—	—	83.6	89.5	91.1	68.2	71.9	80.6	81.2	—	83.8	—	—
SSR2026.06	80.9	—	—	83.6	90.1	92.7	69.3	74.9	82.1	81.4	—	73.3	—	—
BinaryBERT2026.04	80.8	83.9	84.2	91.2	91.4	93.7	53.3	88.6	86	71.5	—	—	—	—
Ours2026.04	80.4	81	81.3	89.2	85.5	92.9	49.3	87.4	85.5	72.2	—	—	—	—
PrefixTuning2025.05	80.18	—	—	87.75	90.77	92.17	55.31	88.48	87.25	54.51	—	85.21	—	—
DOGE TA2026.06	79.9	—	—	83.5	90.3	92.5	69.1	71.1	80.9	79.8	—	71.9	—	—
DELLA2026.05	79.5	—	—	83.8	89.9	91.1	69.3	76	79.9	82	—	64.4	—	—
CMA2026.05	79.4	—	—	73.9	89.4	92.2	70.9	69.7	75.7	80.9	—	82.9	—	—
AdaMerging2026.05	79.4	—	—	81.7	89.8	91.4	69.9	66.1	79.9	79.1	—	77.2	—	—
TA2026.05	78.9	—	—	83.9	89.8	91.7	69.1	73.2	79.4	81.2	—	62.7	—	—
DARE2026.05	78.7	—	—	83.9	89.9	91.7	69.5	69.8	79.7	81.2	—	63.8	—	—
DARE-TIES2026.05	78.6	—	—	83.8	89.8	91.3	69.3	71.1	79.7	81.6	—	62.5	—	—
Weight Averaging2026.06	78.2	—	—	83.8	90.1	91.2	69.7	72	78.9	80.5	—	59.7	—	—
BERT-base2026.06	78.15	—	—	—	—	—	—	—	—	—	—	—	—	—
Concrete TA2026.06	78	—	—	83.5	89.9	91.6	69.1	73.4	78.4	79.4	—	58.1	—	—
TernaryBERT2026.04	77.8	83.3	83.3	90.1	90	92.9	47.8	84.3	82.6	68.4	—	—	—	—
BiT2026.04	77.5	79.5	79.4	85.4	82.5	92.3	38.2	84.2	83	69.7	—	—	—	—
Ties Merging2026.06	77.5	—	—	83.7	89.8	91.6	68.3	71.2	79.4	79.4	—	56.3	—	—
TIES2026.05	77.4	—	—	83.4	89.3	91.3	69.2	68.4	77.7	80.5	—	59.4	—	—
Task Arithmetic2026.06	77.4	—	—	83.7	89.8	91.5	68.8	72.4	78.7	79.1	—	55.2	—	—
moBERTo-8k2026.06	77.24	—	—	—	—	—	—	—	—	—	—	—	—	—
moBERTo2026.06	77.05	—	—	—	—	—	—	—	—	—	—	—	—	—
BiPFT2026.04	76.8	77	76.9	89	86.6	88.1	36.2	87	84.1	66.1	—	—	—	—
MLBERT2026.04	76.3	—	—	—	—	93.5	53.5	—	86.1	72.2	—	—	—	—
Fisher2026.05	76	—	—	83.6	84.6	88.1	69.3	74.4	76.7	77.6	—	54	—	—
RankMean2026.05	75.7	—	—	82.1	88.5	91.2	69.1	62.2	76.2	80.1	—	56.5	—	—
MLBERT2026.04	75.1	—	—	—	—	93.4	52.1	—	85.8	69.2	—	—	—	—
PromptTuning2025.05	74.93	—	—	82.99	80.03	91.09	49.37	82.44	74.83	58.12	—	80.57	—	—
BEBERT2026.04	74.7	75.7	76.6	84.9	80.7	91.8	42.6	—	82.4	65.8	—	—	—	—
NeoBERT-base2026.06	74.3	—	—	—	—	—	—	—	—	—	—	—	—	—
MLBERT2026.04	74	—	—	—	—	92.6	50.2	—	85.2	68.1	—	—	—	—
moBERTo-SWM-8k2026.06	73.54	—	—	—	—	—	—	—	—	—	—	—	—	—
Qwen-base2026.06	72.6	—	—	—	—	—	—	—	—	—	—	—	—	—
moBERTo-SWM2026.06	71.28	—	—	—	—	—	—	—	—	—	—	—	—	—
Qwen3-0.6B-PT2026.06	70.5	—	—	—	—	—	—	—	—	—	—	—	—	—
BERTimbau-base2026.06	67.72	—	—	—	—	—	—	—	—	—	—	—	—	—
BiBERT2026.04	67	66.1	67.5	84.8	76	90.9	37.8	56.7	78.8	61	—	—	—	—
NeoBERT-PT2026.06	66.2	—	—	—	—	—	—	—	—	—	—	—	—	—
RegMean2026.05	65.4	—	—	77.2	79.3	86	69.1	48.1	75.3	61.7	—	26.6	—	—
RFA2026.06	64.9	—	—	86.3	75.8	90.1	27.6	41.2	64.2	53	—	80.8	—	—
Performer2026.06	63.3	—	—	84.4	76.3	88.9	22.8	38.8	66.3	51.6	—	77.5	—	—
TernaryBERT2026.04	58.3	40.3	40	63.1	50	87.5	20.6	72.5	72	47.2	—	—	—	—
BinaryBERT2026.04	52	62.7	63.9	79.9	51	89.6	33	11.4	71	55.9	—	—	—	—
BinaryBERT2026.04	50.1	35.6	35.3	66.2	66.1	78.3	7.3	22.1	69.3	57.7	—	—	—	—
TernaryBERT2026.04	45.5	32.7	33	74.1	50.9	80.3	6.5	10.3	71.5	53.4	—	—	—	—
AdaCubic2026.04	—	81.88	81.89	87.23	90.01	90.71	—	88.32	90.42	70.39	56.33	—	—	—
AdaHessian2026.04	—	81.65	81.57	87.42	89.47	90.71	—	87.86	85.62	64.98	56.33	—	—	—
BERT-Large fine-tuned2026.04	—	—	—	88.08	92.24	93.46	63.35	90.36	91.62	75.45	—	86.64	—	1
SecureRouter + MPC optimized model pool2026.04	—	—	—	88.05	92	93.1	59.1	89.02	91.78	75.09	—	86.73	—	1.53
SGD2026.04	—	82.45	82.05	87.75	90.37	91.62	—	88.63	90.94	70.39	56.33	—	—	—