Pitch Evaluation Dataset
0.0014P-ValueBest Frontier (Gemini 3.1 Pro)
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Best Frontier (Gemini 3.1 Pro)Sample size (N)=115, Statistical Test=McNemar2026.03 | 0.0014 | — | 39.13 | 20.87 | — | — | — | 10.173 | |
| Best Frontier (Gemini 3.1 Pro)N paired=115, Statistical Test=McNemar2026.03 | 0.0014 | — | — | — | — | 39.13 | 20.87 | 10.173 | |
| Junior Majority (full)N paired=103, Statistical Test=McNemar2026.03 | 0.002 | — | — | — | — | 40.78 | 21.36 | 9.587 | |
| Junior majority (full, excl. ties)Sample size (N)=103, Statistical Test=McNemar2026.03 | 0.0029 | — | 40.78 | 20.39 | — | — | — | 8.889 | |
| Expert MajorityN paired=89, Statistical Test=McNemar2026.03 | 0.0073 | — | — | — | — | 41.57 | 21.35 | 7.2 | |
| Expert majority (excl. ties)Sample size (N)=89, Statistical Test=McNemar2026.03 | 0.0104 | — | 41.57 | 20.22 | — | — | — | 6.568 | |
| Frontier average (11 models)Sample size (N)=120, Statistical Test=Exact binomial2026.03 | 1.74 | — | 31.05 | 29.78 | — | — | — | — | |
| Frontier AverageN paired=120, Statistical Test=Exact binomial2026.03 | 1.74 | — | — | — | — | 31.05 | 29.78 | — | |
| SFT 2-Model EnsembleComparison Baseline=Frontier average (11 models), Sample size (N)=1202026.03 | — | 60.83 | — | — | — | — | — | — | |
| SFT 2-Model EnsembleComparison Baseline=Best Frontier (Gemini 3.1 Pro), Sample size (N)=1152026.03 | — | 60 | — | — | — | — | — | — | |
| SFT 2-Model EnsembleComparison Baseline=Expert majority (excl. ties), Sample size (N)=892026.03 | — | 61.8 | — | — | — | — | — | — | |
| SFT 2-Model EnsembleComparison Baseline=Junior majority (full, excl. ties), Sample size (N)=1032026.03 | — | 61.17 | — | — | — | — | — | — | |
| SFT 2-Model ModelComparison Baseline=Frontier Average, N paired=1202026.03 | — | — | — | — | 60.83 | — | — | — | |
| SFT 2-Model ModelComparison Baseline=Best Frontier (Gemini 3.1 Pro), N paired=1152026.03 | — | — | — | — | 60 | — | — | — | |
| SFT 2-Model ModelComparison Baseline=Expert Majority, N paired=892026.03 | — | — | — | — | 62.92 | — | — | — | |
| SFT 2-Model ModelComparison Baseline=Junior Majority (full), N paired=1032026.03 | — | — | — | — | 62.14 | — | — | — |