SuperNI
Benchmarks
TaskDatasetSOTA resultTrendResults
SuperNI (test)
57.35ROUGE Score
72
SuperNI (Order 2)
49.26AP
20
SuperNI (Order 1)
49.48AP
20
SuperNI Hold-In v1.0 (test)
62.47ROUGE-L Score
18
SuperNI Hold-Out v1.0 (test)
53.53ROUGE-L
18
SuperNI Benchmark
50.9Average Score
14
SuperNI Large Number of Tasks (test)
82.1Average Performance
13
SuperNI Standard CL Benchmark (test)
81.9Average Performance
13
SuperNI
56.95AP
13
SuperNI (test)
56.23AP
13
SuperNI Unseen
37.97ROUGE-L
9
SuperNI In-domain
52.26ROUGE-L
9
SuperNI
1.87FWT (O1)
9
SuperNI
48.12Average Accuracy
6
SuperNI Order 2
51.54AP
5
SuperNI (Order 1)
50.84AP
5
SuperNI (unseen tasks)
11.56Dialog Score
4
SuperNI Benchmark
—Primary metric
0