Question Answering on CondaQA (test)
73.3AccuracyUnifiedQA-v2-3B
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| UnifiedQA-v2-3Bnumber of parameters=3B, Training Input=P+Q, Testing Input=P+Q2024.06 | 73.3 | 42.2 | 72.8 | 55.7 | 57.2 | |
| ROBERTa-Large w/ AHBTraining Input=P+Q+AHB, Testing Input=P+Q+AHB2024.06 | 67.1 | 31.4 | 61.9 | 43.8 | 50.7 | |
| UnifiedQA-v2-Largenumber of parameters=770M, Training Input=P+Q, Testing Input=P+Q2024.06 | 66.7 | 30.2 | 64 | 43.7 | 46.5 | |
| ROBERTa-Large w/ AG or ACG (Tested on ACG)Training Input=P+Q+AG or ACG, Testing Input=P+Q+ACG2024.06 | 66.7 | 32.2 | 62.2 | 44.9 | 50.9 | |
| ROBERTa-Large w/ ACGTraining Input=P+Q+ACG, Testing Input=P+Q+ACG2024.06 | 66.4 | 31.7 | 62.6 | 44.6 | 49.4 | |
| ROBERTa-Large w/ SCGTraining Input=P+Q+SCG, Testing Input=P+Q+SCG2024.06 | 65.7 | 28.4 | 60.8 | 42.4 | 48.6 | |
| ROBERTa-Large w/ AG or AHB (Tested on AHB)Training Input=P+Q+AG or AHB, Testing Input=P+Q+AHB2024.06 | 65.7 | 30.2 | 61.1 | 41.3 | 48.9 | |
| ROBERTa-Large w/ AHB+ACGTraining Input=P+Q+AHB+ACG, Testing Input=P+Q+AHB+ACG2024.06 | 65.6 | 30.1 | 60.9 | 43.7 | 49.9 | |
| ROBERTa-Large w/ AG (Trained on AG, Tested on ACG)Training Input=P+Q+AG, Testing Input=P+Q+ACG2024.06 | 65.6 | 30.3 | 61.3 | 42.4 | 49 | |
| ROBERTa-Large w/ STraining Input=P+Q+S, Testing Input=P+Q+S2024.06 | 65.2 | 31.1 | 58.4 | 44.1 | 49.2 | |
| ROBERTa-Large (Our Implementation)number of parameters=355M, Training Input=P+Q, Testing Input=P+Q2024.06 | 64.9 | 29.6 | 61.3 | 42.3 | 48.3 | |
| ROBERTa-Large w/ AG (Trained on AG, Tested on AHB)Training Input=P+Q+AG, Testing Input=P+Q+AHB2024.06 | 64.4 | 28.3 | 57.2 | 40.7 | 46.2 | |
| ROBERTa-Large w/ AG (Training only)Training Input=P+Q+AG, Testing Input=P+Q2024.06 | 63.6 | 26.7 | 61.4 | 38.8 | 43.9 | |
| ROBERTa-Large w/ AHB (Training only)Training Input=P+Q+AHB, Testing Input=P+Q2024.06 | 62.8 | 26.3 | 60.5 | 39.2 | 43.3 | |
| ROBERTa-Large w/ AG or AHB (Training only)Training Input=P+Q+AG or AHB, Testing Input=P+Q2024.06 | 62.5 | 25.7 | 60.1 | 38.6 | 42.4 | |
| ROBERTa-Large w/ ACG (Training only)Training Input=P+Q+ACG, Testing Input=P+Q2024.06 | 61.3 | 23.4 | 59.6 | 37.8 | 37.8 | |
| ROBERTa-Large w/ AG or ACG (Training only)Training Input=P+Q+AG or ACG, Testing Input=P+Q2024.06 | 60.6 | 22 | 57.9 | 35.2 | 36.8 | |
| UnifiedQA-v2-Basenumber of parameters=220M, Training Input=P+Q, Testing Input=P+Q2024.06 | 58 | 17.5 | 54.6 | 30.4 | 33 | |
| ROBERTa-Largenumber of parameters=355M, Training Input=P+Q, Testing Input=P+Q2024.06 | 54.1 | 13.6 | 51.6 | 26.5 | 27.2 |