FAQ-answering on OHO FAQ-answering (test)
0.97PrecisionHUM-oracle
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| HUM-oracleTARA=Gold TARAs2023.06 | 0.97 | 0.98 | 0.93 | 0.73 | 0.95 | 0.83 | |
| HUM2023.06 | 0.89 | 0.87 | 0.67 | 0.61 | 0.76 | 0.72 | |
| QA pattern matching2023.06 | 0.77 | — | 0.18 | — | 0.3 | — | |
| Fine-tuned LMbackbone=PERT, training=two-fold cross-validation2023.06 | 0.57 | 0.64 | 0.26 | 0.23 | 0.36 | 0.34 | |
| Lexical matching2023.06 | 0.45 | 0.5 | 0.56 | 0.46 | 0.5 | 0.48 | |
| Keyword matching2023.06 | 0.45 | 0.54 | 0.74 | 0.41 | 0.56 | 0.52 | |
| Self-supervised LM2023.06 | 0.41 | — | 0.34 | — | 0.37 | — | |
| Semantic matchingbackbone=BERT2023.06 | 0.36 | 0.38 | 0.6 | 0.36 | 0.45 | 0.37 | |
| Pre-trained LMbackbone=PERT2023.06 | 0.24 | — | 0.24 | — | 0.24 | — |