Metric Correlation Analysis on Synthetic perturbation sets (test)
80.27Spearman's rho (S)TABXEVAL
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| TABXEVALMetric Category=LLM-Based, Reference=With Reference2025.12 | 80.27 | 72.37 | 66.87 | 47.54 | 20.94 | 45.33 | |
| TABREXMetric Category=Reference-less, Reference=Reference-less2025.12 | 74.51 | 64.24 | 62.28 | 44.85 | 27.01 | 13.59 | |
| QUESTEVALMetric Category=Reference-less, Reference=Reference-less2025.12 | 62.93 | 52.29 | 51.71 | 42.7 | 35.04 | 3.03 | |
| H-SCOREMetric Category=Non-LLM Based, Reference=With Reference2025.12 | 56.87 | 47.97 | 51.73 | 41.11 | 40.02 | 0.99 | |
| LLM EnsembleEnsemble Type=LLM-based, Aggregation=Harmonic Mean (H)2025.12 | 56 | 46.93 | 50.64 | 40.95 | 40.63 | 0.42 | |
| Hybrid EnsembleEnsemble Type=Reference + Reference-less, Aggregation=Harmonic Mean (H)2025.12 | 54.03 | 42.71 | 32.61 | 42.31 | 40.11 | 1.13 | |
| P-SCOREMetric Category=LLM-Based, Reference=With Reference2025.12 | 49.24 | 40 | 37.43 | 40.73 | 43.93 | 7.39 | |
| TABEVALMetric Category=LLM-Based, Reference=With Reference2025.12 | 49.01 | 39.22 | 34.21 | 41.11 | 43.06 | 0.63 | |
| LLM EnsembleEnsemble Type=LLM-based, Aggregation=Simple Mean (M)2025.12 | 48.49 | 39.21 | 36.94 | 40.56 | 44.38 | 0.42 | |
| EMMetric Category=Non-LLM Based, Reference=With Reference2025.12 | 45.88 | 39.38 | 39.51 | 43.33 | 47.49 | 58.4 | |
| BLUERTMetric Category=Non-LLM Based, Reference=With Reference2025.12 | 44.66 | 37.64 | 36.09 | 39.57 | 48.09 | 0.77 | |
| CHRFMetric Category=Non-LLM Based, Reference=With Reference2025.12 | 41.76 | 34.55 | 31.61 | 39.39 | 49.26 | 1.64 | |
| Lex-EmbEnsemble Type=Lexical + Embedding, Aggregation=Simple Mean (M)2025.12 | 38.43 | 32.65 | 30.17 | 38.52 | 52.15 | 0.49 | |
| BERTSCOREMetric Category=Non-LLM Based, Reference=With Reference2025.12 | 36.21 | 30.66 | 27.96 | 38.11 | 53.25 | 0.92 | |
| Hybrid EnsembleEnsemble Type=Reference + Reference-less, Aggregation=Simple Mean (M)2025.12 | 32.04 | 24.94 | 20.29 | 37.03 | 51.51 | 1.13 | |
| ROUGE-LMetric Category=Non-LLM Based, Reference=With Reference2025.12 | 31.18 | 26.69 | 22.56 | 37.65 | 55.94 | 1.97 | |
| Lex-EmbEnsemble Type=Lexical + Embedding, Aggregation=Harmonic Mean (H)2025.12 | 29.8 | 24 | 19.68 | 37.65 | 55.04 | 0.63 |