Personalized Question Answering on LaMP-QA (test)
49.18Art ScoreIAP(Ours)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| IAP(Ours)Base Model=Mistral-Small 24B2026.05 | 49.18 | 66.49 | 66.28 | — | 60.65 | |
| IAP(Ours)Base Model=Mistral-Small 24B-it2026.05 | 48.11 | 65.05 | 65.36 | — | 59.5 | |
| IAP(Ours)Base Model=Gemma3-12B-it2026.05 | 47.69 | 59.34 | 66.65 | — | 57.89 | |
| IAP(Ours)Base Model=Gemma3-12B2026.05 | 46.12 | 62.18 | 64.3 | — | 57.53 | |
| SFTBase Model=Mistral-Small 24B2026.05 | 43.59 | 62.47 | 64.3 | — | 56.78 | |
| SFTBase Model=Gemma3-12B-it2026.05 | 42.89 | 58.01 | 61.18 | — | 54.02 | |
| Explicit Intent Pers.Base Model=Gemma3-12B-it2026.05 | 42.7 | 59.82 | 60.88 | — | 54.46 | |
| PR2Base LLM=Qwen2.5-7B2026.02 | 42.65 | 55.19 | 59.83 | 52.56 | — | |
| SFTBase Model=Gemma3-12B2026.05 | 42.3 | 61.44 | 61.47 | — | 55.07 | |
| Implicit Intent Pers.Base Model=Gemma3-12B-it2026.05 | 42.11 | 58.72 | 60.44 | — | 53.75 | |
| Explicit Intent Pers.Base Model=Mistral-Small 24B2026.05 | 41.89 | 62.12 | 62.55 | — | 55.52 | |
| SFTBase Model=Mistral-Small 24B-it2026.05 | 41.61 | 58.78 | 61.57 | — | 53.98 | |
| Implicit Intent Pers.Base Model=Mistral-Small 24B2026.05 | 41.58 | 60.33 | 61.84 | — | 54.58 | |
| Explicit Intent Pers.Base Model=Gemma3-12B2026.05 | 41.55 | 61.29 | 61.38 | — | 54.74 | |
| Non-pers.Base Model=Gemma3-12B-it2026.05 | 41.3 | 59.52 | 62.92 | — | 54.58 | |
| Implicit Intent Pers.Base Model=Gemma3-12B2026.05 | 40.78 | 60.34 | 61.2 | — | 54.1 | |
| Explicit Intent Pers.Base Model=Mistral-Small 24B-it2026.05 | 40.47 | 57.88 | 60.72 | — | 53.02 | |
| Non-pers.Base Model=Gemma3-12B2026.05 | 40.33 | 60.05 | 60.48 | — | 53.62 | |
| Implicit Intent Pers.Base Model=Mistral-Small 24B-it2026.05 | 40.28 | 57.38 | 60.49 | — | 52.71 | |
| Non-pers.Base Model=Mistral-Small 24B2026.05 | 40.27 | 58.12 | 59.26 | — | 52.55 | |
| Non-pers.Base Model=Mistral-Small 24B-it2026.05 | 39.66 | 56.1 | 60.03 | — | 51.93 | |
| PR2Base LLM=Gemma3-4B2026.02 | 38.55 | 52.71 | 57.32 | 49.52 | — | |
| PrLMBase LLM=Qwen2.5-7B2026.02 | 38.49 | 48.08 | 54.22 | 46.93 | — | |
| PR2Base LLM=Qwen2.5-3B2026.02 | 37.51 | 49.24 | 57.61 | 48.12 | — | |
| Search R1-GRPOBase LLM=Qwen2.5-7B2026.02 | 36.68 | 49.39 | 54.16 | 46.74 | — | |
| IAP(Ours)Base Model=Qwen2.5-7B2026.05 | 36.29 | 48.89 | 51.78 | — | 45.65 | |
| HYDRA-AdapterBase LLM=Qwen2.5-7B2026.02 | 36.12 | 47.86 | 53.19 | 45.72 | — | |
| Offline RL RAG-PersonalizationRuntime (second / query)=1.672025.08 | 35.79 | 46.21 | 50.7 | 44.23 | — | |
| IAP(Ours)Base Model=Qwen2.5-7B-it2026.05 | 35.33 | 47.18 | 52.4 | — | 44.97 | |
| Search R1-GRPOBase LLM=Gemma3-4B2026.02 | 35.27 | 47.31 | 52.66 | 45.08 | — | |
| PlanPersRuntime (second / query)=3.122025.08 | 35.18 | 48.18 | 52.4 | 45.25 | — | |
| PPlugBase LLM=Qwen2.5-7B2026.02 | 34.88 | 47.73 | 53.01 | 45.2 | — | |
| PPlugBase LLM=Gemma3-4B2026.02 | 34.59 | 43.62 | 51.41 | 43.2 | — | |
| VACRuntime (second / query)=1.632025.08 | 34.54 | 52.88 | 53.31 | 46.91 | — | |
| Search R1-GRPOBase LLM=Qwen2.5-3B2026.02 | 34.11 | 46.91 | 51.4 | 44.14 | — | |
| RAG-PersonalizationRuntime (second / query)=1.712025.08 | 33.97 | 44.81 | 49.67 | 42.81 | — | |
| PPlugBase LLM=Qwen2.5-3B2026.02 | 33.74 | 45.22 | 50.77 | 43.24 | — | |
| HYDRA-AdapterBase LLM=Gemma3-4B2026.02 | 32.71 | 46.25 | 52.73 | 43.89 | — | |
| PrLMBase LLM=Gemma3-4B2026.02 | 32.66 | 48.17 | 53.01 | 44.61 | — | |
| PrLMBase LLM=Qwen2.5-3B2026.02 | 32.28 | 47.62 | 52.76 | 44.22 | — | |
| SFTBase Model=Qwen2.5-7B2026.05 | 32.21 | 45.26 | 48.37 | — | 41.94 | |
| Explicit Intent Pers.Base Model=Qwen2.5-7B2026.05 | 31.69 | 46.58 | 48.03 | — | 42.1 | |
| SFTBase Model=Qwen2.5-7B-it2026.05 | 31.58 | 43.11 | 48.01 | — | 40.9 | |
| Implicit Intent Pers.Base Model=Qwen2.5-7B2026.05 | 31.58 | 46.39 | 47.77 | — | 41.91 | |
| No-PersonalizationRuntime (second / query)=0.782025.08 | 31.29 | 45.82 | 47.69 | 41.6 | — | |
| Non-pers.Base Model=Qwen2.5-7B2026.05 | 31.29 | 45.82 | 47.69 | — | 41.6 | |
| RAG-PersonalizationBase LLM=Qwen2.5-7B2026.02 | 30.86 | 41.07 | 45.9 | 39.28 | — | |
| Explicit Intent Pers.Base Model=Qwen2.5-7B-it2026.05 | 30.32 | 42.66 | 47.66 | — | 40.21 | |
| Implicit Intent Pers.Base Model=Qwen2.5-7B-it2026.05 | 30.04 | 42.06 | 47.29 | — | 39.79 | |
| No personalizationBase LLM=Qwen2.5-7B2026.02 | 29.81 | 40.14 | 43.52 | 37.82 | — | |
| HYDRA-AdapterBase LLM=Qwen2.5-3B2026.02 | 29.42 | 45.04 | 52.11 | 42.19 | — | |
| Non-pers.Base Model=Qwen2.5-7B-it2026.05 | 29.29 | 40.83 | 44.69 | — | 38.27 | |
| No personalizationBase LLM=Gemma3-4B2026.02 | 29.17 | 38.71 | 44.62 | 37.5 | — | |
| RAG-PersonalizationBase LLM=Gemma3-4B2026.02 | 28.87 | 39.65 | 42.34 | 36.95 | — | |
| RAG-PersonalizationBase LLM=Qwen2.5-3B2026.02 | 27.53 | 37.28 | 42.82 | 35.88 | — | |
| RAG-Personalization (Random P)Runtime (second / query)=1.652025.08 | 25.47 | 38.29 | 40.37 | 34.71 | — | |
| No personalizationBase LLM=Qwen2.5-3B2026.02 | 25.03 | 35.11 | 40.73 | 33.62 | — |