Human alignment evaluation on LaMP Citation Identification, Movie Tagging, and Product Rating (test)
100Pos ScoreBLEU
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BLEURate=0.002026.06 | 100 | 83 | 50 | |
| ROUGERate=0.002026.06 | 100 | 81 | 50 | |
| Embedding SimilarityRate=0.002026.06 | 100 | 80 | 50 | |
| NLICV (DeBERTa-Large)Rate=0.002026.06 | 99 | 49 | 99.67 | |
| NLICV (DeBERTa-Large)Rate=0.202026.06 | 98 | 49 | 98.17 | |
| NLICV (DeBERTa-Large)Rate=0.402026.06 | 98 | 49 | 98.5 | |
| NLICV (DeBERTa-Large)Rate=Mean2026.06 | 98 | 49 | 98 | |
| NLICV (DeBERTa-Large)Rate=0.602026.06 | 97 | 50 | 96.67 | |
| NLICV (DeBERTa-Large)Rate=0.802026.06 | 97 | 50 | 97 | |
| NLICV (DeBERTa-Large)Rate=1.002026.06 | 97 | 50 | 98 | |
| NLICV (DeBERTa-Small)Rate=0.002026.06 | 94 | 44 | 96.83 | |
| NLICV (DeBERTa-Small)Rate=0.802026.06 | 94 | 45 | 95.17 | |
| NLICV (DeBERTa-Small)Rate=1.002026.06 | 94 | 44 | 95.5 | |
| NLICV (DeBERTa-Small)Rate=Mean2026.06 | 94 | 44 | 95.78 | |
| NLICV (DeBERTa-Small)Rate=0.202026.06 | 93 | 44 | 95.33 | |
| NLICV (DeBERTa-Small)Rate=0.402026.06 | 93 | 44 | 96.5 | |
| NLICV (DeBERTa-Small)Rate=0.602026.06 | 93 | 45 | 95.33 | |
| Embedding SimilarityRate=0.202026.06 | 84 | 65 | 59.83 | |
| ROUGERate=0.202026.06 | 80 | 63 | 59.67 | |
| BLEURate=0.202026.06 | 71 | 58 | 57.17 | |
| Embedding SimilarityRate=Mean2026.06 | 70 | 53 | 65.75 | |
| NLICV (MiniLM2)Rate=0.002026.06 | 70 | 20 | 68.5 | |
| NLICV (DistilRoBERTa)Rate=0.202026.06 | 70 | 22 | 70.67 | |
| NLICV (RoBERTa)Rate=0.202026.06 | 70 | 22 | 71.17 | |
| NLICV (DistilRoBERTa)Rate=0.002026.06 | 69 | 21 | 68.5 | |
| NLICV (RoBERTa)Rate=0.002026.06 | 69 | 18 | 69 | |
| NLICV (RoBERTa)Rate=0.402026.06 | 69 | 24 | 70.33 | |
| NLICV (RoBERTa)Rate=0.602026.06 | 69 | 25 | 71 | |
| NLICV (RoBERTa)Rate=0.802026.06 | 69 | 26 | 70.17 | |
| NLICV (RoBERTa)Rate=Mean2026.06 | 69 | 24 | 70.11 | |
| NLICV (MiniLM2)Rate=0.202026.06 | 68 | 21 | 66.33 | |
| NLICV (MiniLM2)Rate=Mean2026.06 | 68 | 21 | 67.33 | |
| NLICV (RoBERTa)Rate=1.002026.06 | 68 | 26 | 69 | |
| NLICV (MiniLM2)Rate=0.402026.06 | 67 | 21 | 66.17 | |
| NLICV (MiniLM2)Rate=0.602026.06 | 67 | 21 | 68.33 | |
| NLICV (MiniLM2)Rate=0.802026.06 | 67 | 21 | 67.67 | |
| NLICV (DistilRoBERTa)Rate=0.402026.06 | 67 | 23 | 68.33 | |
| NLICV (DistilRoBERTa)Rate=Mean2026.06 | 67 | 22 | 68.14 | |
| Embedding SimilarityRate=0.402026.06 | 66 | 51 | 71 | |
| NLICV (MiniLM2)Rate=1.002026.06 | 66 | 21 | 67 | |
| NLICV (DistilRoBERTa)Rate=0.602026.06 | 66 | 23 | 67.5 | |
| NLICV (DistilRoBERTa)Rate=0.802026.06 | 66 | 23 | 67.33 | |
| ROUGERate=Mean2026.06 | 65 | 48 | 59.28 | |
| NLICV (DistilRoBERTa)Rate=1.002026.06 | 65 | 23 | 66.5 | |
| ROUGERate=0.402026.06 | 60 | 44 | 72 | |
| Embedding SimilarityRate=0.602026.06 | 58 | 41 | 72.17 | |
| Embedding SimilarityRate=0.802026.06 | 57 | 40 | 71.5 | |
| Embedding SimilarityRate=1.002026.06 | 57 | 41 | 70 | |
| BLEURate=Mean2026.06 | 54 | 42 | 53.89 | |
| ROUGERate=0.602026.06 | 51 | 34 | 58.17 | |
| ROUGERate=0.802026.06 | 50 | 33 | 57.83 | |
| ROUGERate=1.002026.06 | 50 | 33 | 58 | |
| BLEURate=0.402026.06 | 46 | 36 | 61 | |
| BLEURate=0.602026.06 | 37 | 25 | 52.83 | |
| BLEURate=0.802026.06 | 35 | 25 | 51.5 | |
| BLEURate=1.002026.06 | 35 | 25 | 50.83 |