Summarization on GovReport (MAE/PM/Qual Metrics)
17.4Token MAEICL+Feedback
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| ICL+FeedbackLLM=Qwen3-8B2026.01 | 17.4 | 50.55 | 4.76 | 51.17 | 17.29 | 4.89 | 0.1 | 94.81 | 4.88 | |
| FeedbackLLM=Qwen3-8B2026.01 | 22.81 | 58.24 | 4.84 | 21.23 | 47.5 | 4.93 | 0.57 | 79.47 | 4.82 | |
| ICL+FeedbackLLM=LLaMA-3.1-8B-Instruct2026.01 | 25.94 | 38.8 | 4.53 | 22.93 | 35.69 | 4.72 | 0.22 | 82.06 | 4.54 | |
| ICL+FeedbackLLM=Qwen3-4B2026.01 | 26.33 | 34.46 | 4.71 | 26.63 | 39.91 | 4.83 | 0.51 | 58.4 | 4.67 | |
| FeedbackLLM=LLaMA-3.1-8B-Instruct2026.01 | 33.88 | 41.3 | 4.64 | 16.22 | 43.17 | 4.72 | 0.25 | 80.1 | 4.63 | |
| MarkergenLLM=LLaMA-3.1-8B-Instruct2026.01 | 41.45 | 56.67 | 3.11 | 34.2 | 54.5 | 3.54 | 3.36 | 31.22 | 3.66 | |
| MarkergenLLM=Qwen3-8B2026.01 | 44.26 | 37.27 | 4.29 | 42.19 | 20.91 | 4.62 | 1.87 | 38.61 | 4.74 | |
| BaselineLLM=Qwen3-8B2026.01 | 52.04 | 15.2 | 4.96 | 60.61 | 6.06 | 4.95 | 3.83 | 35.59 | 4.91 | |
| BaselineLLM=Qwen3-4B2026.01 | 58.19 | 11.94 | 4.94 | 70.74 | 8.31 | 4.95 | 15.85 | 16.36 | 4.74 | |
| ICLLLM=Qwen3-4B2026.01 | 59.07 | 13.3 | 4.77 | 104.2 | 3.88 | 4.8 | 8.2 | 18.41 | 4.53 | |
| ICLLLM=Qwen3-8B2026.01 | 59.51 | 11.16 | 4.76 | 141.2 | 0.16 | 4.92 | 4.09 | 27.98 | 4.89 | |
| FeedbackLLM=Qwen3-4B2026.01 | 61.3 | 21.35 | 4.64 | 31.96 | 38.53 | 4.84 | 1.38 | 47.83 | 4.62 | |
| BaselineLLM=LLaMA-3.1-8B-Instruct2026.01 | 108.65 | 5.93 | 4.78 | 32.83 | 14.32 | 4.81 | 2.94 | 46.64 | 4.58 | |
| ICLLLM=LLaMA-3.1-8B-Instruct2026.01 | 118.27 | 3.31 | 4.7 | 50.6 | 8.66 | 4.79 | 3.32 | 37.77 | 4.54 |