Alphabet-size estimation on Pooled QA Datasets (SQuAD, CoQA, NQ-Open, TriviaQA, HotpotQA)
1.14MAESHADE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SHADEsubsample size (n)=50, oracle size (N)=1002026.04 | 1.14 | 1.86 | |
| Hybrid (GT+EigV)subsample size (n)=50, oracle size (N)=1002026.04 | 1.38 | 2.23 | |
| Hybrid-GGTsubsample size (n)=50, oracle size (N)=1002026.04 | 1.38 | 2.21 | |
| Generalized GT (GGT)subsample size (n)=50, oracle size (N)=1002026.04 | 1.42 | 2.3 | |
| Good-Turing (GT)subsample size (n)=50, oracle size (N)=1002026.04 | 1.43 | 2.32 | |
| Plugin (kobs)subsample size (n)=50, oracle size (N)=1002026.04 | 1.54 | 2.62 | |
| SHADEsubsample size (n)=25, oracle size (N)=1002026.04 | 2.05 | 3.36 | |
| Hybrid-GGTsubsample size (n)=25, oracle size (N)=1002026.04 | 2.27 | 3.6 | |
| Hybrid (GT+EigV)subsample size (n)=25, oracle size (N)=1002026.04 | 2.28 | 3.63 | |
| U-EigV (Laplacian)subsample size (n)=50, oracle size (N)=1002026.04 | 2.47 | 3.93 | |
| Generalized GT (GGT)subsample size (n)=25, oracle size (N)=1002026.04 | 2.48 | 3.81 | |
| Good-Turing (GT)subsample size (n)=25, oracle size (N)=1002026.04 | 2.5 | 3.85 | |
| Plugin (kobs)subsample size (n)=25, oracle size (N)=1002026.04 | 2.66 | 4.24 | |
| SHADEsubsample size (n)=8, oracle size (N)=1002026.04 | 2.83 | 4.37 | |
| SHADEsubsample size (n)=5, oracle size (N)=1002026.04 | 2.86 | 4.82 | |
| U-EigV (Laplacian)subsample size (n)=25, oracle size (N)=1002026.04 | 2.88 | 4.59 | |
| SHADEsubsample size (n)=10, oracle size (N)=1002026.04 | 3.01 | 4.25 | |
| Hybrid (GT+EigV)subsample size (n)=10, oracle size (N)=1002026.04 | 3.06 | 4.7 | |
| Hybrid-GGTsubsample size (n)=10, oracle size (N)=1002026.04 | 3.06 | 4.76 | |
| Hybrid-GGTsubsample size (n)=8, oracle size (N)=1002026.04 | 3.2 | 5.03 | |
| Hybrid (GT+EigV)subsample size (n)=8, oracle size (N)=1002026.04 | 3.22 | 4.99 | |
| Good-Turing (GT)subsample size (n)=10, oracle size (N)=1002026.04 | 3.3 | 4.86 | |
| Generalized GT (GGT)subsample size (n)=10, oracle size (N)=1002026.04 | 3.3 | 4.96 | |
| Hybrid-GGTsubsample size (n)=5, oracle size (N)=1002026.04 | 3.4 | 5.42 | |
| Generalized GT (GGT)subsample size (n)=8, oracle size (N)=1002026.04 | 3.43 | 5.22 | |
| Good-Turing (GT)subsample size (n)=8, oracle size (N)=1002026.04 | 3.44 | 5.12 | |
| U-EigV (Laplacian)subsample size (n)=10, oracle size (N)=1002026.04 | 3.51 | 5.49 | |
| Hybrid (GT+EigV)subsample size (n)=5, oracle size (N)=1002026.04 | 3.52 | 5.52 | |
| Generalized GT (GGT)subsample size (n)=5, oracle size (N)=1002026.04 | 3.6 | 5.57 | |
| Plugin (kobs)subsample size (n)=10, oracle size (N)=1002026.04 | 3.65 | 5.6 | |
| Good-Turing (GT)subsample size (n)=5, oracle size (N)=1002026.04 | 3.66 | 5.55 | |
| U-EigV (Laplacian)subsample size (n)=8, oracle size (N)=1002026.04 | 3.66 | 5.73 | |
| Plugin (kobs)subsample size (n)=8, oracle size (N)=1002026.04 | 3.83 | 5.87 | |
| U-EigV (Laplacian)subsample size (n)=5, oracle size (N)=1002026.04 | 3.98 | 6.16 | |
| Plugin (kobs)subsample size (n)=5, oracle size (N)=1002026.04 | 4.18 | 6.34 |