Scholarly Abstract Generation on LaMP Scholar
44.6ROUGE-1RankGPT (GPT-5-nano)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| RankGPT (GPT-5-nano)LLM Backbone=Llama-3-70B-Instruct (70.6B)2026.01 | 44.6 | 38.8 | 41.5 | |
| PURPLELLM Backbone=Llama-3-70B-Instruct (70.6B)2026.01 | 44.4 | 38.8 | 41 | |
| ICR (Llama-3-8B-Instruct)LLM Backbone=Llama-3-70B-Instruct (70.6B)2026.01 | 44.3 | 38.5 | 41.2 | |
| ContrieverLLM Backbone=Llama-3-70B-Instruct (70.6B)2026.01 | 44.1 | 38.6 | 41.1 | |
| RankGPT (Llama-3-8B-Instruct)LLM Backbone=Llama-3-70B-Instruct (70.6B)2026.01 | 44 | 38.6 | 41.1 | |
| BM25LLM Backbone=Llama-3-70B-Instruct (70.6B)2026.01 | 43.8 | 38.2 | 40.5 | |
| REPLUGLLM Backbone=Llama-3-8B-Instruct (8.03B)2026.01 | 42.5 | 37.1 | 40.8 | |
| REPLUGLLM Backbone=Llama-3-70B-Instruct (70.6B)2026.01 | 41.8 | 36.6 | 38.6 | |
| PURPLELLM Backbone=Llama-3-8B-Instruct (8.03B)2026.01 | 41.4 | 35.8 | 40.3 | |
| ICR (Llama-3-8B-Instruct)LLM Backbone=Llama-3-8B-Instruct (8.03B)2026.01 | 41.3 | 35.5 | 40.8 | |
| ContrieverLLM Backbone=Llama-3-8B-Instruct (8.03B)2026.01 | 41.1 | 35.4 | 40.5 | |
| BM25LLM Backbone=Llama-3-8B-Instruct (8.03B)2026.01 | 41 | 35.2 | 40.6 | |
| RankGPT (Llama-3-8B-Instruct)LLM Backbone=Llama-3-8B-Instruct (8.03B)2026.01 | 41 | 35.4 | 40.5 | |
| RankGPT (GPT-5-nano)LLM Backbone=Llama-3-8B-Instruct (8.03B)2026.01 | 41 | 35.3 | 40.5 | |
| PURPLELLM Backbone=Phi-4-Mini-Instruct (3.84B)2026.01 | 40 | 33.5 | 42.4 | |
| RankGPT (GPT-5-nano)LLM Backbone=Phi-4-Mini-Instruct (3.84B)2026.01 | 39.8 | 33.4 | 42.3 | |
| BM25LLM Backbone=Phi-4-Mini-Instruct (3.84B)2026.01 | 39.7 | 33.2 | 42.2 | |
| ContrieverLLM Backbone=Phi-4-Mini-Instruct (3.84B)2026.01 | 39.7 | 33.3 | 42 | |
| RankGPT (Llama-3-8B-Instruct)LLM Backbone=Phi-4-Mini-Instruct (3.84B)2026.01 | 39.7 | 33.3 | 42 | |
| IC-RALMLLM Backbone=Llama-3-70B-Instruct (70.6B)2026.01 | 39.7 | 34.1 | 38.6 | |
| ICR (Llama-3-8B-Instruct)LLM Backbone=Phi-4-Mini-Instruct (3.84B)2026.01 | 39.5 | 33.1 | 42 | |
| REPLUGLLM Backbone=Phi-4-Mini-Instruct (3.84B)2026.01 | 38.1 | 32.8 | 40.4 | |
| IC-RALMLLM Backbone=Phi-4-Mini-Instruct (3.84B)2026.01 | 37.6 | 31 | 41 | |
| IC-RALMLLM Backbone=Llama-3-8B-Instruct (8.03B)2026.01 | 37.5 | 31.5 | 39.2 |