Insight Generation on SCOpE-QA (Set-level)
4.45Inference Optimization ScoreINSIGHTGEN
Evaluation Results
| Method | Links | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| INSIGHTGENGenerator Model=Claude-3.5-Sonnet, Judge Model=Gemini-1.5-Flash2026.04 | 4.45 | 4.44 | 4.31 | 4.41 | 4.33 | 4.06 | 4.01 | 4.39 | 4.53 | 4.36 | 4.38 | 3.97 | 4.34 | 4.32 | 4.51 | 4.28 | 4.54 | 4.5 | 4.37 | 4.07 | |
| FAISS+CoTGenerator Model=Claude-3.5-Sonnet, Judge Model=Gemini-1.5-Flash2026.04 | 3.98 | 4.08 | 4.07 | 4.17 | 3.94 | 4.33 | 4.07 | 3.68 | 4.24 | 4.19 | 4.04 | 3.79 | 3.95 | 4.08 | 4.19 | 3.9 | 3.79 | 3.89 | 4.03 | 3.69 | |
| FAISSGenerator Model=GPT-4o, Judge Model=Gemini-1.5-Flash2026.04 | 3.87 | 3.44 | 3.68 | 3.81 | 3.55 | 3.73 | 3.21 | 3.62 | 3.84 | 3.45 | 3.76 | 3.84 | 3.33 | 3.47 | 3.73 | 3.65 | 3.69 | 3.75 | 3.92 | 3.93 | |
| INSIGHTGENGenerator Model=GPT-4o, Judge Model=Gemini-1.5-Flash2026.04 | 3.87 | 4.14 | 4.1 | 4.15 | 4.14 | 4.15 | 4.52 | 4.39 | 4.48 | 4.23 | 4.34 | 4.39 | 4.36 | 4.45 | 4.31 | 4.35 | 4.27 | 4.27 | 3.98 | 4.26 | |
| DirectGenerator Model=GPT-4o, Judge Model=Gemini-1.5-Flash2026.04 | 3.79 | 2.99 | 3.12 | 3.61 | 3.2 | 3.49 | 2.82 | 3.51 | 2.91 | 3.24 | 3.35 | 3.01 | 3.22 | 2.99 | 3.15 | 2.75 | 3.06 | 2.88 | 3.59 | 2.71 | |
| GPT+CoTGenerator Model=Claude-3.5-Sonnet, Judge Model=Gemini-1.5-Flash2026.04 | 3.32 | 2.31 | 2.98 | 3.54 | 2.5 | 3.26 | 3.57 | 2.92 | 2.59 | 2.86 | 3.27 | 3.17 | 2.37 | 2.55 | 2.31 | 3.21 | 2.05 | 1.56 | 2.57 | 2.23 | |
| DirectGenerator Model=Claude-3.5-Sonnet, Judge Model=Gemini-1.5-Flash2026.04 | 3.18 | 2.23 | 2.95 | 2.29 | 2.37 | 2.32 | 2.67 | 2.85 | 1.79 | 2.24 | 2.95 | 3.56 | 3.17 | 3.13 | 2.15 | 2.86 | 2.11 | 1.63 | 2.11 | 2.28 | |
| INSIGHTGENGenerator Model=Claude-3.5-Sonnet, Judge Model=Claude-4-Sonnet2026.04 | 3.11 | 3.46 | 3.24 | 3.33 | 3.27 | 2.86 | 3.14 | 3.07 | 3.45 | 3.15 | 3.35 | 3.19 | 3.41 | 3.14 | 3.53 | 3.46 | 3.52 | 3.46 | 3.09 | 3.21 | |
| FAISSGenerator Model=Claude-3.5-Sonnet, Judge Model=Gemini-1.5-Flash2026.04 | 3.09 | 2.53 | 2.59 | 2.81 | 3.13 | 3.12 | 2.75 | 3.12 | 2.77 | 3.05 | 3.09 | 3.01 | 3.19 | 3.27 | 2.73 | 2.85 | 2.97 | 3.55 | 3.32 | 3.46 | |
| FAISSGenerator Model=GPT-4o, Judge Model=Claude-4-Sonnet2026.04 | 3.06 | 2.56 | 2.74 | 2.86 | 2.63 | 2.64 | 2.6 | 2.66 | 2.66 | 2.66 | 2.86 | 3.01 | 2.49 | 2.57 | 2.92 | 2.86 | 2.91 | 2.68 | 2.78 | 2.86 | |
| INSIGHTGENGenerator Model=GPT-4o, Judge Model=Claude-4-Sonnet2026.04 | 3.05 | 3.37 | 3.32 | 3.41 | 3.47 | 3.22 | 3.84 | 3.43 | 3.52 | 3.57 | 3.6 | 3.81 | 3.81 | 3.36 | 3.53 | 3.66 | 3.53 | 3.56 | 2.89 | 3.63 | |
| FAISS+CoTGenerator Model=GPT-4o, Judge Model=Gemini-1.5-Flash2026.04 | 3.03 | 3.12 | 3.37 | 3.25 | 2.93 | 3.14 | 3.06 | 2.97 | 3.3 | 3.03 | 2.85 | 2.93 | 2.85 | 2.97 | 3.09 | 3.26 | 3.07 | 3.29 | 3.15 | 3 | |
| GPT+CoTGenerator Model=GPT-4o, Judge Model=Gemini-1.5-Flash2026.04 | 3.02 | 3.25 | 2.22 | 3.2 | 2.81 | 2.66 | 3.27 | 2.82 | 2.76 | 3.13 | 2.8 | 2.71 | 2.95 | 2.65 | 2.54 | 3.3 | 2.71 | 1.9 | 2.2 | 2.11 | |
| DirectGenerator Model=GPT-4o, Judge Model=Claude-4-Sonnet2026.04 | 2.79 | 2.04 | 2 | 2.76 | 2.16 | 2.67 | 2.16 | 2.6 | 1.97 | 2.24 | 2.2 | 2.08 | 2.05 | 0.96 | 2.38 | 2.04 | 2.39 | 2.08 | 2.96 | 2.06 | |
| DirectGenerator Model=Claude-3.5-Sonnet, Judge Model=Claude-4-Sonnet2026.04 | 2.69 | 1.77 | 2.31 | 1.67 | 1.68 | 1.97 | 1.81 | 2.64 | 1.57 | 2.27 | 2.21 | 2.62 | 2.33 | 2.17 | 1.77 | 2.11 | 1.83 | 1.02 | 2.61 | 1.99 | |
| FAISSGenerator Model=Claude-3.5-Sonnet, Judge Model=Claude-4-Sonnet2026.04 | 2.61 | 1.89 | 2.26 | 2.21 | 2.45 | 2.5 | 2.17 | 2.35 | 2.2 | 2.45 | 2.52 | 2.47 | 2.47 | 2.51 | 2.11 | 2.25 | 2.48 | 2.72 | 2.44 | 2.69 | |
| FAISS+CoTGenerator Model=Claude-3.5-Sonnet, Judge Model=Claude-4-Sonnet2026.04 | 2.29 | 2.71 | 2.63 | 2.85 | 2.24 | 2.92 | 2.38 | 2.02 | 2.85 | 2.44 | 2.49 | 2.57 | 2.45 | 2.55 | 2.8 | 2.45 | 2.45 | 2.43 | 2.4 | 2.2 | |
| GPT+CoTGenerator Model=Claude-3.5-Sonnet, Judge Model=Claude-4-Sonnet2026.04 | 2.27 | 1.2 | 2.03 | 2.56 | 1.27 | 2.15 | 1.98 | 1.95 | 1.55 | 1.67 | 1.58 | 2.07 | 1.77 | 1.17 | 1.41 | 2.35 | 1.22 | 0.85 | 1.99 | 1.23 | |
| FAISS+CoTGenerator Model=GPT-4o, Judge Model=Claude-4-Sonnet2026.04 | 2.06 | 2.16 | 2.35 | 2.3 | 2.05 | 2.22 | 2.24 | 2.12 | 2.42 | 2.07 | 2.13 | 2.07 | 2.43 | 2.13 | 2.38 | 2.55 | 2.18 | 2.29 | 2.09 | 1.95 | |
| GPT+CoTGenerator Model=GPT-4o, Judge Model=Claude-4-Sonnet2026.04 | 1.92 | 2.32 | 1.28 | 2.06 | 1.86 | 1.53 | 2.26 | 2.05 | 2.24 | 2.21 | 1.82 | 1.94 | 1.71 | 0.96 | 1.76 | 2.03 | 1.91 | 1.14 | 1.24 | 1.65 |