Relational Reasoning on CLUTRR
0.69Empirical Generalization Error G(W)CoLA
Evaluation Results
| Method | Links | |
|---|---|---|
| CoLABackbone=TinyLlama [71]2026.06 | 0.69 | |
| r-cLABackbone=Llama 3 [15]2026.06 | 0.94 | |
| FFTBackbone=Llama 3 [15]2026.06 | 0.95 | |
| cLABackbone=TinyLlama [71]2026.06 | 0.96 | |
| c3LABackbone=TinyLlama [71]2026.06 | 0.97 | |
| LoRABackbone=TinyLlama [71]2026.06 | 1 | |
| LoRABackbone=Llama 3 [15]2026.06 | 1 | |
| LoRA+Backbone=Llama 3 [15]2026.06 | 1 | |
| RACBackbone=TinyLlama [71]2026.06 | 1.01 | |
| cLABackbone=Llama 3 [15]2026.06 | 1.01 | |
| AsymmBackbone=TinyLlama [71]2026.06 | 1.04 | |
| AsymmBackbone=Llama 3 [15]2026.06 | 1.09 | |
| CoLABackbone=Llama 3 [15]2026.06 | 1.12 | |
| r-cLABackbone=TinyLlama [71]2026.06 | 1.15 | |
| CoLAModel=TinyLlama2026.06 | 1.55 | |
| CoLAModel (Backbone)=TinyLlama [71]2026.06 | 1.55 | |
| r-c3LABackbone=Llama 3 [15]2026.06 | 1.63 | |
| r-c3LABackbone=TinyLlama [71]2026.06 | 1.88 | |
| c3LABackbone=Llama 3 [15]2026.06 | 1.89 | |
| FFTBackbone=TinyLlama [71]2026.06 | 1.91 | |
| RACBackbone=Llama 3 [15]2026.06 | 2.06 | |
| cLAModel=TinyLlama2026.06 | 2.16 | |
| cLAModel (Backbone)=TinyLlama [71]2026.06 | 2.16 | |
| c3LAModel=TinyLlama2026.06 | 2.19 | |
| c3LAModel (Backbone)=TinyLlama [71]2026.06 | 2.19 | |
| LoRAModel=TinyLlama2026.06 | 2.25 | |
| LoRAModel (Backbone)=TinyLlama [71]2026.06 | 2.25 | |
| RACModel=TinyLlama2026.06 | 2.27 | |
| RACModel (Backbone)=TinyLlama [71]2026.06 | 2.27 | |
| AsymModel=TinyLlama2026.06 | 2.34 | |
| AsymmModel (Backbone)=TinyLlama [71]2026.06 | 2.34 | |
| LoRA+Backbone=TinyLlama [71]2026.06 | 2.44 | |
| r-cLAModel=Llama3-8B2026.06 | 2.51 | |
| r-cLAModel (Backbone)=Llama 3 [15]2026.06 | 2.51 | |
| FFTModel=Llama3-8B2026.06 | 2.53 | |
| FFTModel (Backbone)=Llama 3 [15]2026.06 | 2.53 | |
| r-cLAModel=TinyLlama2026.06 | 2.59 | |
| r-cLAModel (Backbone)=TinyLlama [71]2026.06 | 2.59 | |
| LoRA+Model=Llama3-8B2026.06 | 2.65 | |
| LoRA+Model (Backbone)=Llama 3 [15]2026.06 | 2.65 | |
| LoRAModel=Llama3-8B2026.06 | 2.66 | |
| LoRAModel (Backbone)=Llama 3 [15]2026.06 | 2.66 | |
| cLAModel=Llama3-8B2026.06 | 2.69 | |
| cLAModel (Backbone)=Llama 3 [15]2026.06 | 2.69 | |
| AsymModel=Llama3-8B2026.06 | 2.9 | |
| AsymmModel (Backbone)=Llama 3 [15]2026.06 | 2.9 | |
| CoLAModel=Llama3-8B2026.06 | 2.97 | |
| CoLAModel (Backbone)=Llama 3 [15]2026.06 | 2.97 | |
| r-c3LAModel=TinyLlama2026.06 | 4.23 | |
| r-c3LAModel (Backbone)=TinyLlama [71]2026.06 | 4.23 | |
| FFTModel=TinyLlama2026.06 | 4.29 | |
| FFTModel (Backbone)=TinyLlama [71]2026.06 | 4.29 | |
| r-c3LAModel=Llama3-8B2026.06 | 4.33 | |
| r-c3LAModel (Backbone)=Llama 3 [15]2026.06 | 4.33 | |
| c3LAModel=Llama3-8B2026.06 | 5.02 | |
| c3LAModel (Backbone)=Llama 3 [15]2026.06 | 5.02 | |
| LoRA+Model=TinyLlama2026.06 | 5.48 | |
| LoRA+Model (Backbone)=TinyLlama [71]2026.06 | 5.48 | |
| RACModel=Llama3-8B2026.06 | 5.49 | |
| RACModel (Backbone)=Llama 3 [15]2026.06 | 5.49 |