Conversational Question Answering on ShARC Long (dev)
72.4Easy AccuracyDOCHOPPER
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DOCHOPPERquery update=w/o query update2021.06 | 72.4 | 59.9 | |
| DOCHOPPERruntime=164.3/s2021.06 | 72.3 | 60.2 | |
| DOCHOPPERnumber of hops=single-hop2021.06 | 68 | 52.7 | |
| Retrieval + DISCERNruntime=42.0/s2021.06 | 65.2 | 50.7 | |
| Sequential (DISCERN)runtime=8.8/s2021.06 | 63.7 | 54.2 | |
| DOCHOPPERinput granularity=sentence only2021.06 | 62.2 | 43.2 | |
| ETCruntime=11.2/s2021.06 | 61.1 | — |