Metric-Human Agreement Evaluation on Human-agreement n=150 (test)
0.502Delta-Kappa (Hybrid-EX vs Baseline)Spider2 M6
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Spider2 M6Bootstrap iterations (N)=5,000, Sample size (n)=1502026.06 | 0.502 | 0.384 | 0.001 | — | |
| Dr. Spider M5Bootstrap iterations (N)=5,000, Sample size (n)=1502026.06 | 0.482 | 0.312 | 0.001 | — | |
| Defog M1Bootstrap iterations (N)=5,000, Sample size (n)=1502026.06 | 0.365 | 0.211 | 0.001 | — | |
| IBM EvalAssist M2Bootstrap iterations (N)=5,000, Sample size (n)=1502026.06 | 0.362 | 0.211 | 0.001 | — | |
| RAGAS M4Bootstrap iterations (N)=5,000, Sample size (n)=1502026.06 | 0.347 | 0.197 | 0.001 | — | |
| BIRD M3Bootstrap iterations (N)=5,000, Sample size (n)=1502026.06 | 0.322 | 0.174 | 0.001 | — | |
| LLM-EXBootstrap iterations (N)=5,000, Sample size (n)=1502026.06 | 0.096 | 0.066 | 0.245 | — |