Human-Metric Correlation on CoGym (Out-of-Distribution)
0.365Kendall's TauAutoMetrics
Evaluation results
| Method | Links | |
|---|---|---|
| AutoMetricsBackbone=Qwen-3-32B2025.12 | 0.365 | |
| DnA EvalBackbone=Qwen-3-32B2025.12 | 0.353 | |
| LLM-JudgeBackbone=Qwen-3-32B2025.12 | 0.276 | |
| Finetuned LLMBackbone=Model Agnostic2025.12 | 0.223 | |
| LLM-JudgeBackbone=GPT-4o-mini2025.12 | 0.199 | |
| DnA EvalBackbone=GPT-4o-mini2025.12 | 0.185 | |
| Best Existing MetricBackbone=Model Agnostic2025.12 | 0.074 | |
| AutoMetricsBackbone=GPT-4o-mini2025.12 | -0.034 | |
| MetaMetricsBackbone=Model Agnostic2025.12 | -0.119 |