Code-Switched Question Answering on CodeMixQA
17.53Avg. F1Adaptive CANVAS (Adapt)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Adaptive CANVAS (Adapt)Backbone=Qwen3.6-35B-A3B, L=40, Lctrl=(28, 39)2026.06 | 17.53 | 0.66 | 3.58 | 1.41 | 1.28 | |
| Random CANVAS (Rand)Backbone=Qwen3.6-35B-A3B, L=40, Lctrl=(28, 39)2026.06 | 16.07 | — | — | — | — | |
| Greedy Decoding (BASE)Backbone=Qwen3.6-35B-A3B, L=40, Lctrl=(28, 39)2026.06 | 15.8 | — | — | — | — | |
| Adaptive CANVAS (Adapt)Backbone=Qwen3.5-27B, L=48, Lctrl=(33, 47)2026.06 | 15.4 | 0.28 | 1.74 | 0.96 | 2.03 | |
| Random CANVAS (Rand)Backbone=Qwen3.5-27B, L=48, Lctrl=(33, 47)2026.06 | 14.22 | — | — | — | — | |
| Greedy Decoding (BASE)Backbone=Qwen3.5-27B, L=48, Lctrl=(33, 47)2026.06 | 14.14 | — | — | — | — | |
| Adaptive CANVAS (Adapt)Backbone=Llama3.3-70B, L=80, Lctrl=(56, 79)2026.06 | 13.61 | 0.29 | 1.66 | 0.61 | 0.04 | |
| Random CANVAS (Rand)Backbone=Llama3.3-70B, L=80, Lctrl=(56, 79)2026.06 | 13.27 | — | — | — | — | |
| Adaptive CANVAS (Adapt)Backbone=Llama3.1-70B, L=80, Lctrl=(56, 79)2026.06 | 13.2 | -0.52 | 0.67 | 0.73 | 0.35 | |
| Random CANVAS (Rand)Backbone=Llama3.1-70B, L=80, Lctrl=(56, 79)2026.06 | 12.98 | — | — | — | — | |
| Greedy Decoding (BASE)Backbone=Llama3.3-70B, L=80, Lctrl=(56, 79)2026.06 | 12.96 | — | — | — | — | |
| Greedy Decoding (BASE)Backbone=Llama3.1-70B, L=80, Lctrl=(56, 79)2026.06 | 12.89 | — | — | — | — | |
| Adaptive CANVAS (Adapt)Backbone=Qwen3.5-9B, L=40, Lctrl=(28, 39)2026.06 | 10.24 | -0.2 | 0.77 | 0.22 | 0.61 | |
| Random CANVAS (Rand)Backbone=Qwen3.5-9B, L=40, Lctrl=(28, 39)2026.06 | 10.01 | — | — | — | — | |
| Greedy Decoding (BASE)Backbone=Qwen3.5-9B, L=40, Lctrl=(28, 39)2026.06 | 9.89 | — | — | — | — | |
| Adaptive CANVAS (Adapt)Backbone=Phi-3.5-MoE, L=32, Lctrl=(22, 31)2026.06 | 8.82 | 0.12 | 1.31 | 0.61 | 1.26 | |
| Adaptive CANVAS (Adapt)Backbone=Qwen3.5-4B, L=36, Lctrl=(25, 35)2026.06 | 8.14 | 0.37 | 0.7 | 0.33 | 0.86 | |
| Adaptive CANVAS (Adapt)Backbone=Qwen3-30B-A3B, L=48, Lctrl=(33, 47)2026.06 | 8.11 | 0.06 | 2.48 | 0.71 | 1.55 | |
| Greedy Decoding (BASE)Backbone=Phi-3.5-MoE, L=32, Lctrl=(22, 31)2026.06 | 7.99 | — | — | — | — | |
| Random CANVAS (Rand)Backbone=Phi-3.5-MoE, L=32, Lctrl=(22, 31)2026.06 | 7.93 | — | — | — | — | |
| Adaptive CANVAS (Adapt)Backbone=Llama3.1-8B, L=32, Lctrl=(22, 31)2026.06 | 7.79 | 1.21 | 0.64 | 2.15 | 2.34 | |
| Greedy Decoding (BASE)Backbone=Qwen3.5-4B, L=36, Lctrl=(25, 35)2026.06 | 7.58 | — | — | — | — | |
| Random CANVAS (Rand)Backbone=Qwen3.5-4B, L=36, Lctrl=(25, 35)2026.06 | 7.35 | — | — | — | — | |
| Random CANVAS (Rand)Backbone=Qwen3-30B-A3B, L=48, Lctrl=(33, 47)2026.06 | 6.93 | — | — | — | — | |
| Adaptive CANVAS (Adapt)Backbone=Mixtral-8x7B, L=32, Lctrl=(22, 31)2026.06 | 6.91 | 0.16 | 0.29 | 0.38 | -0.29 | |
| Greedy Decoding (BASE)Backbone=Qwen3-30B-A3B, L=48, Lctrl=(33, 47)2026.06 | 6.91 | — | — | — | — | |
| Adaptive CANVAS (Adapt)Backbone=Aya-Expanse-8B, L=32, Lctrl=(22, 31)2026.06 | 6.88 | 0.62 | 1.41 | 0.63 | 0.91 | |
| Random CANVAS (Rand)Backbone=Mixtral-8x7B, L=32, Lctrl=(22, 31)2026.06 | 6.82 | — | — | — | — | |
| Greedy Decoding (BASE)Backbone=Mixtral-8x7B, L=32, Lctrl=(22, 31)2026.06 | 6.77 | — | — | — | — | |
| Random CANVAS (Rand)Backbone=Llama3.1-8B, L=32, Lctrl=(22, 31)2026.06 | 6.72 | — | — | — | — | |
| Greedy Decoding (BASE)Backbone=Llama3.1-8B, L=32, Lctrl=(22, 31)2026.06 | 6.21 | — | — | — | — | |
| Greedy Decoding (BASE)Backbone=Aya-Expanse-8B, L=32, Lctrl=(22, 31)2026.06 | 5.99 | — | — | — | — | |
| Random CANVAS (Rand)Backbone=Aya-Expanse-8B, L=32, Lctrl=(22, 31)2026.06 | 5.99 | — | — | — | — | |
| Adaptive CANVAS (Adapt)Backbone=Mistral-7B, L=32, Lctrl=(22, 31)2026.06 | 5.06 | 0.14 | 0.8 | 0.65 | 0.55 | |
| Random CANVAS (Rand)Backbone=Mistral-7B, L=32, Lctrl=(22, 31)2026.06 | 4.96 | — | — | — | — | |
| Greedy Decoding (BASE)Backbone=Mistral-7B, L=32, Lctrl=(22, 31)2026.06 | 4.53 | — | — | — | — | |
| Adaptive CANVAS (Adapt)Backbone=Llama3.2-1B, L=16, Lctrl=(11, 15)2026.06 | 3.2 | 0.08 | 0.82 | 1.01 | 0.62 | |
| Random CANVAS (Rand)Backbone=Llama3.2-1B, L=16, Lctrl=(11, 15)2026.06 | 2.68 | — | — | — | — | |
| Greedy Decoding (BASE)Backbone=Llama3.2-1B, L=16, Lctrl=(11, 15)2026.06 | 2.57 | — | — | — | — |