Commonsense Reasoning on BoolQ (Inter/Intra-Layer Filtering)
67Accuracy (Inter-Layer Filtering)CRFT
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| CRFTBackbone=LLaMA-3-8B, Identify=MSF2025.07 | 67 | 67.9 | — | |
| CRFTBackbone=LLaMA-3-8B, Identify=SSF2025.07 | 64 | 66.6 | — | |
| CRFTBackbone=LLaMA-3-8B, Identify=SAF2025.07 | 63 | 66.4 | — | |
| CRFTBackbone=Mistral-7B, Identify=SAF2025.07 | 63 | 66.5 | — | |
| ReFTBackbone=Mistral-7B, Identify=p7+s72025.07 | 62.5 | 62.5 | — | |
| CRFTBackbone=Mistral-7B, Identify=MSF2025.07 | 62.5 | 65 | — | |
| CRFTBackbone=LLaMA-3-8B, Identify=MAF2025.07 | 62.4 | 66.2 | — | |
| CRFTBackbone=Mistral-7B, Identify=SSF2025.07 | 62.3 | 64.8 | — | |
| ReFTBackbone=LLaMA-3-8B, Identify=p7+s72025.07 | 62.1 | 62.1 | — | |
| CRFTBackbone=Mistral-7B, Identify=MAF2025.07 | 62.1 | 60.8 | — | |
| CRFTBackbone=LLaMA-2-7B, Identify=SSF2025.07 | 62 | 54.3 | — | |
| CRFTBackbone=LLaMA-2-7B, Identify=MAF2025.07 | 60.5 | 61.8 | — | |
| CRFTBackbone=LLaMA-2-7B, Identify=SAF2025.07 | 60 | 53.7 | — | |
| CRFTBackbone=LLaMA-2-7B, Identify=MSF2025.07 | 60 | 59.7 | — | |
| ReFTBackbone=LLaMA-2-7B, Identify=p7+s72025.07 | 50.7 | 50.7 | — | |
| ArrowZero-shot=true, Base Model=Phi-32025.05 | — | — | 80 | |
| GenKnowSubSetting=En, Zero-shot=true, Base Model=Phi-32025.05 | — | — | 81.96 | |
| GenKnowSubSetting=De, Zero-shot=true, Base Model=Phi-32025.05 | — | — | 82.01 | |
| GenKnowSubSetting=Fr, Zero-shot=true, Base Model=Phi-32025.05 | — | — | 82.11 | |
| GenKnowSubSetting=Avg, Zero-shot=true, Base Model=Phi-32025.05 | — | — | 82.45 | |
| Mean NormZero-shot=true, Base Model=Phi-32025.05 | — | — | 80.89 | |
| Phi-3Zero-shot=true, Base Model=Phi-32025.05 | — | — | 81.47 | |
| SharedZero-shot=true, Base Model=Phi-32025.05 | — | — | 63.39 |