Question Answering on SQuAD (F1, Subspan EM, EM Count)

71.4F1 ScoreResults with Original Prompts

Evaluation results

Method
Results with Original Prompts2024.09	71.4	99.93	2,805	47.49
TACO-RL - with F12024.09	69.62	99.92	2,736	46.32
TACO-RL - with F1 + Token Wise Score2024.09	68.03	99.9	2,637	44.64
TACO-RL - with F12024.09	65.84	99.92	2,592	43.88
TACO-RL - with F1 + Token Wise Score2024.09	62.79	99.81	2,412	40.83
LLMLingua-2 - Wikitext2024.09	62.71	99.86	2,231	37.77
LLMLingua-2 - Squad2024.09	62.7	99.83	2,246	38.02
TACO-RL - with F12024.09	61.7	99.88	2,386	40.39
TACO-RL - with F1 + Token Wise Score2024.09	58.37	99.71	2,195	37.16
TACO-RL - with F12024.09	57.92	99.76	2,199	37.23
TACO-RL - with Token Wise Score2024.09	56.67	99.78	2,076	35.14
TACO-RL - with F12024.09	55.46	99.81	2,067	34.99
TACO-RL - with F1 + Token Wise Score2024.09	54.57	99.78	2,024	34.26
LLMLingua-2 - Squad2024.09	53.11	99.58	1,838	31.12
LLMLingua-2 - Wikitext2024.09	52.86	99.66	1,848	31.28
TACO-RL - with F1 + Token Wise Score2024.09	51.56	99.64	1,867	31.61
Selective Context2024.09	51.39	98.68	2,042	34.57
Selective Context2024.09	46.37	98.26	1,813	30.69
LLMLingua-2 - Squad2024.09	46.37	99.27	1,548	26.21
TACO-RL - with Token Wise Score2024.09	46.25	99.31	1,636	27.7
LLMLingua-2 - Wikitext2024.09	46.15	99.61	1,545	26.16
LLMLingua-2 - Squad2024.09	41.97	99.12	1,391	23.55
LLMLingua-2 - Wikitext2024.09	41.49	99.41	1,363	23.07
Selective Context2024.09	40.35	97.9	1,555	26.32
TACO-RL - with Token Wise Score2024.09	39.85	99.17	1,342	22.72
LLMLingua-2 - Wikitext2024.09	38.78	99.27	1,259	21.31
LLMLingua-2 - Squad2024.09	38.74	98.98	1,268	21.47
Selective Context2024.09	37.37	97.83	1,417	23.99
LLMLingua2024.09	36.62	98.87	1,108	18.76
Selective Context2024.09	35.53	97.82	1,323	22.4
TACO-RL - with Token Wise Score2024.09	35.2	98.97	1,167	19.76
TACO-RL - with Token Wise Score2024.09	31.98	98.93	1,011	17.12
LLMLingua2024.09	30.57	97.85	974	16.49
LLMLingua2024.09	28.27	97.58	918	15.54
LLMLingua2024.09	26.96	97.26	866	14.66
LLMLingua2024.09	25.96	97.14	848	14.36