Human label alignment on Human expert binary labels n = 150
0.717KappaHybrid-EX
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Hybrid-EXClassification=Substantial, Rank=#12026.06 | 0.717 | 0.6 | 87.3 | 81.5 | 93.1 | 11.6 | |
| LLM-EXClassification=Substantial, Rank=#22026.06 | 0.621 | 0.489 | 78.8 | 98.9 | 58.6 | 40.3 | |
| BIRD M3Classification=Fair, Rank=#32026.06 | 0.395 | 0.265 | 72 | 54.4 | 89.7 | 35.3 | |
| RAGAS M4Classification=Fair, Rank=#42026.06 | 0.37 | 0.226 | 70.2 | 57.6 | 82.8 | 25.2 | |
| IBM EvalAssist M2Classification=Fair, Rank=#52026.06 | 0.355 | 0.209 | 69.3 | 57.6 | 81 | 23.4 | |
| Defog M1Classification=Fair, Rank=#62026.06 | 0.351 | 0.209 | 69.4 | 54.4 | 84.5 | 30.1 | |
| Dr. Spider M5Classification=Fair (low), Rank=#72026.06 | 0.235 | 0.113 | 60.1 | 97.8 | 22.4 | 75.4 | |
| Spider2 M6Classification=Fair (low), Rank=#82026.06 | 0.214 | 0.132 | 63 | 26.1 | 100 | 73.9 |