Commonsense Reasoning on Commonsense Reasoning Suite (test)

0.9594HellaSwag AccuracyPERA (r = 16)

Evaluation Results

Method
PERA (r = 16)2026.04	0.9594	0.8738	—	0.8857	0.9373	0.8362	—	—	0.886	—	0.8927	0.7615	0.8316	—
PERA (r = 32)2026.04	0.9561	0.8712	—	0.8983	0.9356	0.8353	—	—	0.884	—	0.8895	0.7554	0.8158	—
DoRA (r = 32)2026.04	0.955	0.852	—	0.893	0.905	0.804	—	—	0.858	—	0.856	0.746	0.799	—
PERA (r = 16)2026.04	0.9538	0.8829	—	0.8961	0.9617	0.8891	—	—	0.918	—	0.8934	0.7306	0.8204	—
HiRA (r = 32)2026.04	0.9536	0.8672	—	0.897	0.9327	0.829	—	—	0.8832	—	0.877	0.754	0.8115	—
HiRA (r = 16)2026.04	0.9485	0.8608	—	0.8912	0.9306	0.8259	—	—	0.874	—	0.8674	0.7385	0.8106	—
LoRA (r = 16)2026.04	0.935	0.828	—	0.867	0.877	0.757	—	—	0.828	—	0.848	0.723	0.793	—
HiRA (r = 16)2026.04	0.923	0.854	—	0.883	0.954	0.887	—	—	0.88	—	0.81	0.69	0.808	—
LoRA (r = 32)2026.04	0.917	0.8079	—	0.852	0.842	0.712	—	—	0.79	—	0.843	0.708	0.799	—
DoRA (r = 32)2026.04	0.891	0.7969	—	0.837	0.837	0.682	—	—	0.824	—	0.826	0.718	0.76	—
PERA (r = 32)2026.04	0.8898	0.8214	—	0.8439	0.8779	0.7329	—	—	0.832	—	0.8627	0.737	0.7953	—
PERA (r = 16)2026.04	0.8866	0.8261	—	0.8531	0.8855	0.7355	—	—	0.854	—	0.8713	0.7183	0.8035	—
HiRA (r = 32)2026.04	0.8812	0.8142	—	0.8335	0.8674	0.7381	—	—	0.846	—	0.8398	0.7122	0.7953	—
LoRA (r = 32)2026.04	0.836	0.7761	—	0.799	0.798	0.647	—	—	0.81	—	0.826	0.698	0.795	—
Dense2025.12	0.8105	0.714	—	0.8226	0.8093	0.5392	—	—	0.442	—	0.738	0.8364	—	—
Qwen2.5 7B-NL2025.05	0.81	—	—	0.84	—	—	—	—	—	—	0.62	—	—	0.95
LoRA (r = 16)2026.04	0.786	0.738	—	0.736	0.859	0.717	—	—	0.744	—	0.758	0.6	0.7	—
ChatGPT2026.04	0.785	0.7701	—	0.854	0.898	0.799	—	—	0.748	—	0.661	0.731	0.685	—
Token Filtering2025.12	0.7664	0.6582	—	0.7661	0.7614	0.4573	—	—	0.414	—	0.693	0.7498	—	—
Dense w/o2026.06	0.7602	0.6436	—	0.7911	0.763	0.4633	—	—	0.442	—	0.6914	0.7771	0.4606	—
ShortGPT2026.06	0.7369	0.6316	—	0.7715	0.7256	0.4445	—	—	0.42	—	0.6914	0.7869	0.4759	—
BUDDY2026.06	0.7364	0.6263	—	0.7709	0.7302	0.448	—	—	0.43	—	0.6882	0.733	0.4734	—
SLEB2026.06	0.726	0.6088	—	0.7748	0.734	0.4104	—	—	0.414	—	0.6401	0.7196	0.4514	—
Shortened Llama2026.06	0.7253	0.6084	—	0.7802	0.7327	0.4283	—	—	0.406	—	0.629	0.7125	0.4529	—
POTARA2026.02	0.7173	0.7418	—	0.788	0.878	0.752	—	—	0.7847	—	0.6453	0.6787	0.69	—
Llama 3.1 8B-NL2025.05	0.71	—	—	0.77	—	—	—	—	—	—	0.66	—	—	0.82
Llama 3.1 8B-PC2025.05	0.71	—	—	0.82	—	—	—	—	—	—	0.61	—	—	0.89
PuDDing2026.06	0.6973	0.5948	—	0.7595	0.6932	0.3951	—	—	0.4	—	0.6448	0.7355	0.433	—
Local2026.02	0.6893	0.7205	—	0.7727	0.862	0.7213	—	—	0.754	—	0.6407	0.6627	0.6613	—
FedALT2026.02	0.6833	0.7148	—	0.79	0.85	0.7273	—	—	0.7667	—	0.6033	0.6427	0.6553	—
Granite 3.1 8B-PC2025.05	0.68	—	—	0.78	—	—	—	—	—	—	0.7	—	—	0.92
Olmo 3 7B-NL2025.05	0.68	—	—	0.72	—	—	—	—	—	—	0.57	—	—	0.92
ShortGPT2026.06	0.6766	0.572	—	0.7242	0.6528	0.3891	—	—	0.396	—	0.6709	0.6294	0.4371	—
BUDDY2026.06	0.6664	0.5844	—	0.7318	0.6528	0.3805	—	—	0.392	—	0.6685	0.7263	0.4565	—
Olmo 3 7B-PC2025.05	0.66	—	—	0.74	—	—	—	—	—	—	0.55	—	—	0.92
FedDPA2026.02	0.6573	0.7233	—	0.804	0.876	0.7473	—	—	0.7707	—	0.6147	0.638	0.6787	—
SLEB2026.06	0.6572	0.5678	—	0.7378	0.6658	0.3549	—	—	0.398	—	0.6046	0.6924	0.4319	—
FedSA2026.02	0.6493	0.7115	—	0.7773	0.8467	0.7153	—	—	0.7487	—	0.6467	0.6427	0.6653	—
FedIT2026.02	0.6493	0.7194	—	0.788	0.8593	0.7427	—	—	0.7433	—	0.6053	0.6687	0.6987	—
SlimGPT w/o2025.12	0.6467	0.5551	—	0.7176	0.5295	0.3439	—	—	0.346	—	0.644	0.6581	—	—
Shortened Llama2026.06	0.6315	0.5377	—	0.7427	0.6326	0.349	—	—	0.37	—	0.5462	0.6187	0.411	—
Mistral 7B v0.3-PC2025.05	0.62	—	—	0.77	—	—	—	—	—	—	0.6	—	—	0.8
Token Filtering2025.12	0.6119	0.5037	—	0.6899	0.4928	0.343	—	—	0.32	—	0.6369	0.4315	—	—
PuDDing2026.06	0.603	0.5307	—	0.7182	0.6141	0.3558	—	—	0.366	—	0.5667	0.6287	0.393	—
ShortGPT2026.06	0.5876	0.5331	—	0.6692	0.5446	0.3413	—	—	0.332	—	0.6409	0.7122	0.4365	—
FLAP2025.12	0.5806	0.5772	—	0.6899	0.67	0.3763	—	—	0.386	—	0.6519	0.6859	—	—
Qwen2.5 7B-CP2025.05	0.58	—	—	0.83	—	—	—	—	—	—	0.62	—	—	0.96
BUDDY2026.06	0.5787	0.5354	—	0.7078	0.5808	0.3114	—	—	0.356	—	0.6022	0.7281	0.4181	—
FFA-LoRA2026.02	0.5773	0.688	—	0.7907	0.8553	0.7327	—	—	0.7387	—	0.5413	0.648	0.62	—
SLEB2026.06	0.5483	0.5093	—	0.7138	0.5976	0.3106	—	—	0.36	—	0.5296	0.6028	0.4115	—
Mistral 7B v0.3-NL2025.05	0.53	—	—	0.65	—	—	—	—	—	—	0.4	—	—	0.74
Shortened Llama2026.06	0.5281	0.4992	—	0.7089	0.5728	0.2986	—	—	0.342	—	0.5233	0.6214	0.3987	—
Qwen2.5 7B-PC2025.05	0.52	—	—	0.84	—	—	—	—	—	—	0.56	—	—	0.95
BUDDY2026.06	0.4866	0.469	—	0.6474	0.4891	0.2841	—	—	0.328	—	0.5351	0.6021	0.3792	—
FiRST2026.06	0.4766	0.4691	—	0.562	0.4827	0.3473	—	—	0.36	—	0.5422	0.5774	0.4048	—
PuDDing2026.06	0.4736	0.4494	—	0.6469	0.4916	0.2935	—	—	0.316	—	0.5359	0.4642	0.3736	—
PuDDing2026.06	0.4736	0.4491	—	0.6447	0.4916	0.2935	—	—	0.316	—	0.5359	0.4642	0.3736	—
ShortGPT2026.06	0.4713	0.4641	—	0.6181	0.4491	0.2918	—	—	0.298	—	0.5722	0.622	0.39	—
Granite 8B Code-NL2025.05	0.47	—	—	0.63	—	—	—	—	—	—	0.54	—	—	0.71
FiRST2026.06	0.4452	0.4397	—	0.5522	0.452	0.3294	—	—	0.354	—	0.4038	0.5774	0.4038	—
SlimGPT w/o2025.12	0.4445	0.432	—	0.6413	0.3096	0.2364	—	—	0.312	—	0.5221	0.5587	—	—
FiRST2026.06	0.4399	0.4512	—	0.5517	0.4533	0.3089	—	—	0.362	—	0.5367	0.5587	0.3982	—
MoRA (r = 32)2026.04	0.4353	0.7863	—	0.8743	0.9116	0.7961	—	—	0.856	—	0.8674	0.7428	0.8071	—
SLEB2026.06	0.4293	0.46	—	0.6507	0.4949	0.2637	—	—	0.31	—	0.5257	0.6171	0.3884	—
FLAP2025.12	0.4052	0.4457	—	0.5911	0.4481	0.2477	—	—	0.314	—	0.5616	0.5524	—	—
Granite 8B Code-PC2025.05	0.4	—	—	0.54	—	—	—	—	—	—	0.51	—	—	0.54
FiRST2026.06	0.3973	0.4215	—	0.5321	0.4112	0.2927	—	—	0.332	—	0.5328	0.5453	0.3291	—
Residual2026.01	0.366	0.462	0.354	0.654	0.584	0.264	0.63	0.366	0.326	0.575	0.52	0.44	—	—
KromHC2026.01	0.364	0.477	0.384	0.65	0.578	0.276	0.66	0.3	0.312	0.586	0.548	0.584	—	—
mHC2026.01	0.362	0.445	0.376	0.642	0.586	0.274	0.64	0.324	0.336	0.604	0.532	0.546	—	—
Shortened Llama2026.06	0.3546	0.4517	—	0.6551	0.4971	0.2662	—	—	0.308	—	0.513	0.6217	0.3976	—
mHC-lite2026.01	0.352	0.444	0.362	0.646	0.586	0.272	0.63	0.236	0.304	0.586	0.492	0.416	—	—
MoRA (r = 32)2026.04	0.2909	0.7246	—	0.8079	0.8531	0.7142	—	—	0.812	—	0.8019	0.7217	0.7953	—
KromHC2026.01	0.28	0.424	0.278	0.59	0.414	0.23	0.54	0.288	0.254	0.535	0.5	0.608	—	—
mHC-lite2026.01	0.274	0.423	0.282	0.564	0.428	0.244	0.53	0.35	0.256	0.534	0.5	0.544	—	—
Residual2026.01	0.27	0.411	0.29	0.604	0.428	0.218	0.55	0.318	0.238	0.593	0.492	0.374	—	—
mHC2026.01	0.264	0.422	0.284	0.608	0.428	0.224	0.6	0.23	0.248	0.553	0.504	0.544	—	—
Mistral 7B v0.3-CP2025.05	0.25	—	—	0.76	—	—	—	—	—	—	0.45	—	—	0.87
Llama 3.1 8B-CP2025.05	0.24	—	—	0.76	—	—	—	—	—	—	0.44	—	—	0.9
Granite 8B Code-CP2025.05	0.24	—	—	0.58	—	—	—	—	—	—	0.47	—	—	0.6
Granite 3.1 8B-CP2025.05	0.23	—	—	0.73	—	—	—	—	—	—	0.43	—	—	0.9
Prompt Tuning2026.04	0.1401	0.3696	—	0.4505	0.3274	0.3157	—	—	0.292	—	0.5012	0.5685	0.3613	—
Granite 3.1 8B-NL2025.05	0.14	—	—	0.4	—	—	—	—	—	—	0.38	—	—	0.16
Olmo 3 7B-CP2025.05	0.07	—	—	0.72	—	—	—	—	—	—	0.49	—	—	0.91
Prompt Tuning2026.04	0.0691	0.2129	—	0.1235	0.0863	0.0606	—	—	0.094	—	0.4057	0.5593	0.305	—
P-Tuning2026.04	0.0177	0.1811	—	0.1164	0.0863	0.0742	—	—	0.096	—	0.3765	0.5997	0.0819	—
P-Tuning2026.04	0.0001	0.1224	—	0.3602	0.0198	0.0017	—	—	0.008	—	0	0.5875	0.002	—