Meeting Summarization on QMSum
4.47CoherenceGemma-3 w/ S3
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Gemma-3 w/ S3Granularity=Segment-level, # of Params=12B2026.06 | 4.47 | 4.68 | 4.98 | 4.83 | 4.74 | 15.1 | |
| Gemma-3 w/ S2Granularity=Segment-level, # of Params=12B2026.06 | 4.41 | 4.66 | 4.97 | 4.83 | 4.72 | 15.2 | |
| Qwen2.5-Instruct w/ S3Granularity=Segment-level, # of Params=7B2026.06 | 4.22 | 4.43 | 4.94 | 4.65 | 4.56 | 14.8 | |
| Qwen2.5-InstructGranularity=Summary-level, # of Params=72B2026.06 | 4.19 | 4.43 | 4.95 | 4.58 | 4.54 | 25.6 | |
| Gemma-3 w/ RefineGranularity=Summary-level, # of Params=12B2026.06 | 4.14 | 4.27 | 4.96 | 4.48 | 4.46 | 24.2 | |
| Qwen2.5-Instruct w/ S2Granularity=Segment-level, # of Params=7B2026.06 | 4.1 | 4.33 | 4.91 | 4.55 | 4.47 | 17.4 | |
| Gemma-3Granularity=Summary-level, # of Params=12B2026.06 | 4.07 | 4.31 | 4.92 | 4.41 | 4.43 | 25.4 | |
| Qwen2.5-InstructGranularity=Summary-level, # of Params=7B2026.06 | 4.01 | 4.22 | 4.9 | 4.36 | 4.37 | 28.7 | |
| Qwen2.5-Instruct w/ RefineGranularity=Summary-level, # of Params=7B2026.06 | 4 | 4.18 | 4.93 | 4.34 | 4.36 | 28.5 | |
| Reference summaryGranularity=-, # of Params=-2026.06 | 2.62 | 3 | 4.02 | 2.87 | 3.13 | — |