Personalization on LaMP-2
67.9AccPRISP
Evaluation results
| Method | Links | ||
|---|---|---|---|
| PRISPRegime=FULL, Backbone=Llama-3.1-8B-Instruct2026.01 | 67.9 | 59.1 | |
| OPPURegime=FULL, Backbone=Llama-3.1-8B-Instruct2026.01 | 64.3 | 56.1 | |
| RPMBackbone=GPT-4o-mini, +CoT=false, Reasoning generation=enabled2025.05 | 56.1 | 46.3 | |
| RAGBackbone=GPT-4o-mini, +CoT=false2025.05 | 52.6 | 43.8 | |
| HYDRABackbone=GPT-4o-mini, +CoT=false2025.05 | 52.6 | 43.7 | |
| FermiBackbone=GPT-4o-mini, +CoT=false2025.05 | 52.6 | 43.7 | |
| PAGBackbone=GPT-4o-mini, +CoT=false2025.05 | 52.5 | 44.4 | |
| PAGBackbone=GPT-4o-mini, +CoT=true2025.05 | 51.3 | 43.1 | |
| RPMBackbone=GPT-4o-mini, +CoT=false, Reasoning generation=disabled2025.05 | 51 | 39.8 | |
| HYDRABackbone=GPT-4o-mini, +CoT=true2025.05 | 49.6 | 40.6 | |
| ICLBackbone=GPT-4o-mini, +CoT=false2025.05 | 49.5 | 41.2 | |
| RAGBackbone=GPT-4o-mini, +CoT=true2025.05 | 49.3 | 41.5 | |
| PRISPRegime=FEW, Backbone=Llama-3.1-8B-Instruct2026.01 | 49.1 | 40.7 | |
| FermiBackbone=GPT-4o-mini, +CoT=true2025.05 | 47.6 | 37.7 | |
| ICLBackbone=GPT-4o-mini, +CoT=true2025.05 | 47.1 | 37.4 | |
| Zero-shotBackbone=GPT-4o-mini, +CoT=false2025.05 | 43 | 36 | |
| Zero-shotBackbone=GPT-4o-mini, +CoT=true2025.05 | 41.1 | 33.7 | |
| PRiMERegime=FULL, Backbone=Llama-3.1-8B-Instruct2026.01 | 37.4 | 35 | |
| Per-PcsRegime=FULL, Backbone=Llama-3.1-8B-Instruct2026.01 | 36.8 | 34.1 | |
| OPPURegime=FEW, Backbone=Llama-3.1-8B-Instruct2026.01 | 23.6 | 19.8 | |
| PRiMERegime=FEW, Backbone=Llama-3.1-8B-Instruct2026.01 | 20.5 | 16.1 | |
| Per-PcsRegime=FEW, Backbone=Llama-3.1-8B-Instruct2026.01 | 20 | 15.6 |