TEMPREASON
Benchmarks
TaskDatasetSOTA resultTrendResults
TempReason-L3 in-domain (test)
0.851EM
20
TempReason-L2 in-domain (test)
80.8EM
20
TempReason OBQA-L3
49.8Exact Match (EM)
17
TempReason OBQA-L2
48EM
17
TempReason
84.75Accuracy
13
TEMPREASON 1.0 (test)
8,110EM
12
TEMPREASON 1.0 (test)
8,480EM
12
TempReason Structured
100Accuracy
4
TEMPREASON 1.0 (test)
100EM
4