Automated Metric Evaluation on Scientific QA LitQA2 (n=58)
1CoverageDynRub-FT
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| DynRub-FTmode=single-pass, temperature=02026.06 | 1 | 0.35 | 0 | 0.27 | 0.27 | 0.95 | 0.27 | 0.07 | 0.27 | 0.04 | |
| Litmusmode=single-pass, temperature=02026.06 | 1 | 0.32 | 0.01 | 0.71 | 0.72 | 0.94 | 0.72 | 0.43 | 0.71 | 0.001 | |
| DynRub-INSTmode=single-pass, temperature=02026.06 | 0.83 | 0.15 | 0 | 0.06 | 0.06 | 0.95 | 0.06 | 0.12 | 0.06 | 1 | |
| AutoMetricsmode=single-pass, temperature=02026.06 | 0.67 | 0.19 | 0.14 | 0.68 | 0.69 | 0.92 | 0.47 | 0.24 | 0.42 | 0.001 | |
| DynRub-DSmode=single-pass, temperature=02026.06 | 0.67 | 0.35 | 0 | 0.12 | 0.12 | 0.95 | 0.12 | 0.23 | 0.12 | 1 |