Continual Model Refinement for Extractive QA on MRQA streams (val) benchmark leaderboard · SOTA2 Research

Continual Model Refinement for Extractive QA on MRQA streams (val)

97.49EFROnline EWC

Evaluation Results

Method	Links
Online EWC2022.05		97.49	73.38	86.09	56.17	47.34	65.75	68.55	85.74	53.67	53.28	65.31
MaxLoss2022.05		97.43	75.43	86.89	57.14	46.7	66.54	75	84.77	55.11	51.33	66.55
MaxLoss2022.05		97.41	75.57	87.09	56.8	46.45	66.48	77.54	89.65	55.88	52.81	68.97
Continual Fine-Tuning2022.05		97.36	72.05	83.87	55.93	45.68	64.38	66.21	77.73	53.48	48.91	61.58
Online L2Reg.2022.05		97.18	73.47	85.37	57.27	47.12	65.81	71.09	83.59	54.5	51.17	65.09
MIR2022.05		97.08	75.92	87.13	56.91	47.22	66.79	75.78	87.5	54.53	51.8	67.4
Experience Replay2022.05		97.07	75.3	87.29	56.02	47.61	66.55	72.46	87.3	54.08	52.66	66.63
MIR2022.05		96.96	77.86	89.41	58.13	46.4	67.95	79.69	89.45	55.5	50.08	68.68
Experience Replay2022.05		96.72	78.91	89.38	57.8	47.17	68.31	78.13	86.52	55.33	52.73	68.18
MIR2022.05		96.71	77.47	87.83	57.98	46.87	67.54	78.13	87.89	55.73	50.7	68.11
MaxLoss2022.05		96.63	77.61	89.82	58.13	47.1	68.17	80.66	91.02	55.88	50.78	69.59
MIR2022.05		96.59	77.84	89.77	58.35	47.28	68.31	79.49	90.43	55.91	51.25	69.27
MaxLoss2022.05		96.54	78.16	89.86	57.78	46.63	68.11	77.54	89.26	55.47	50.94	68.3
MIR(1,256)+OnlineL2Reg2022.05		96.15	79.1	90.41	59.8	47.9	69.3	79.49	90.04	57.45	52.66	69.91
Offline Refining2022.05		95.62	—	—	—	—	—	83.78	93.75	93.81	56.17	81.88
Frozen Upstream2022.05		0	80.27	43.69	44.95	31.25	50.04	80.27	36.13	35.44	31.25	45.77