Question Answering on TriviaQA (ACC, ASR)
75ACCSDAG-D&A
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| SDAG-D&A#adv. docs=1, k=10, Generator=Llama, Retriever=E52026.02 | 75 | 10 | |
| SDAG#adv. docs=1, k=10, Generator=Llama, Retriever=E52026.02 | 74 | 11 | |
| SDAG-D&A#adv. docs=1, k=5, Generator=Llama, Retriever=E52026.02 | 71 | 17 | |
| SDAG#adv. docs=1, k=5, Generator=Llama, Retriever=E52026.02 | 69 | 20 | |
| SDAG-RAGD#adv. docs=1, k=10, Generator=Llama, Retriever=E52026.02 | 69 | 11 | |
| RAGD#adv. docs=1, k=10, Generator=Llama, Retriever=E52026.02 | 68 | 22 | |
| RAGD#adv. docs=3, k=10, Generator=Llama, Retriever=E52026.02 | 67 | 26 | |
| SDAG-RAGD#adv. docs=2, k=10, Generator=Llama, Retriever=E52026.02 | 66 | 14 | |
| RAGD#adv. docs=2, k=10, Generator=Llama, Retriever=E52026.02 | 65 | 24 | |
| SDAG-RAGD#adv. docs=3, k=10, Generator=Llama, Retriever=E52026.02 | 65 | 18 | |
| RAGD#adv. docs=2, k=5, Generator=Llama, Retriever=E52026.02 | 64 | 25 | |
| SDAG-D&A#adv. docs=2, k=10, Generator=Llama, Retriever=E52026.02 | 63 | 28 | |
| D&A#adv. docs=1, k=10, Generator=Llama, Retriever=E52026.02 | 62 | 35 | |
| top-k decodingModel=Phi-3-medium-4k-instruct, Temp=0.3, lambda (λ)=0.1, alpha (α)=22025.05 | 60.5 | — | |
| top-k decodingModel=Phi-3-medium-4k-instruct, Temp=0.7, lambda (λ)=0.1, alpha (α)=22025.05 | 60.5 | — | |
| top-k decodingModel=Phi-3-medium-4k-instruct, Temp=1, lambda (λ)=0.1, alpha (α)=22025.05 | 60.5 | — | |
| top-k decodingModel=Phi-3-medium-4k-instruct, Temp=1.5, lambda (λ)=0.1, alpha (α)=22025.05 | 60.5 | — | |
| SDAG-RAGD#adv. docs=2, k=5, Generator=Llama, Retriever=E52026.02 | 60 | 23 | |
| Bregman primal decodingModel=Phi-3-medium-4k-instruct, Temp=0.3, lambda (λ)=0.1, alpha (α)=22025.05 | 59.67 | — | |
| top-k decodingModel=Phi-3-medium-4k-instruct, Temp=0.3, lambda (λ)=0.01, alpha (α)=22025.05 | 59.39 | — | |
| Bregman primal decodingModel=Phi-3-medium-4k-instruct, Temp=0.3, lambda (λ)=0.01, alpha (α)=1.52025.05 | 59.33 | — | |
| Bregman primal decodingModel=Phi-3-medium-4k-instruct, Temp=0.3, lambda (λ)=0.01, alpha (α)=22025.05 | 59.22 | — | |
| top-k decodingModel=Phi-3-medium-4k-instruct, Temp=0.3, lambda (λ)=0.01, alpha (α)=1.52025.05 | 59.11 | — | |
| top-k decodingModel=Phi-3-medium-4k-instruct, Temp=0.3, lambda (λ)=0.1, alpha (α)=1.52025.05 | 59.05 | — | |
| D&A#adv. docs=1, k=5, Generator=Llama, Retriever=E52026.02 | 59 | 40 | |
| SDAG-RAGD#adv. docs=1, k=5, Generator=Llama, Retriever=E52026.02 | 59 | 25 | |
| SDAG#adv. docs=2, k=10, Generator=Llama, Retriever=E52026.02 | 59 | 29 | |
| Bregman primal decodingModel=Phi-3-medium-4k-instruct, Temp=0.3, lambda (λ)=0.1, alpha (α)=1.52025.05 | 58.44 | — | |
| Bregman primal decodingModel=Phi-3-medium-4k-instruct, Temp=0.7, lambda (λ)=0.1, alpha (α)=22025.05 | 58.22 | — | |
| Bregman primal decodingModel=Phi-3-medium-4k-instruct, Temp=1.5, lambda (λ)=0.1, alpha (α)=22025.05 | 58.22 | — | |
| Bregman primal decodingModel=Phi-3-medium-4k-instruct, Temp=0.7, lambda (λ)=0.1, alpha (α)=1.52025.05 | 57.44 | — | |
| Bregman primal decodingModel=Phi-3-medium-4k-instruct, Temp=1.5, lambda (λ)=0.1, alpha (α)=1.52025.05 | 57.16 | — | |
| CARG#adv. docs=1, k=10, Generator=Llama, Retriever=E52026.02 | 57 | 41 | |
| Bregman primal decodingModel=Phi-3-medium-4k-instruct, Temp=1, lambda (λ)=0.1, alpha (α)=22025.05 | 56.94 | — | |
| top-k decodingModel=Phi-3-medium-4k-instruct, Temp=0.7, lambda (λ)=0.1, alpha (α)=1.52025.05 | 56.77 | — | |
| top-k decodingModel=Phi-3-medium-4k-instruct, Temp=0.7, lambda (λ)=0.01, alpha (α)=22025.05 | 56.77 | — | |
| Bregman primal decodingModel=Phi-3-medium-4k-instruct, Temp=1, lambda (λ)=0.1, alpha (α)=1.52025.05 | 56.6 | — | |
| D&A#adv. docs=2, k=10, Generator=Llama, Retriever=E52026.02 | 56 | 47 | |
| Bregman primal decodingModel=Phi-3-medium-4k-instruct, Temp=0.7, lambda (λ)=0.01, alpha (α)=22025.05 | 55.88 | — | |
| top-k decodingModel=Phi-3-medium-4k-instruct, Temp=0.7, lambda (λ)=0.01, alpha (α)=1.52025.05 | 55.54 | — | |
| Bregman primal decodingModel=Phi-3-medium-4k-instruct, Temp=0.7, lambda (λ)=0.01, alpha (α)=1.52025.05 | 55.21 | — | |
| top-k decodingModel=Phi-3-medium-4k-instruct, Temp=1, lambda (λ)=0.1, alpha (α)=1.52025.05 | 54.54 | — | |
| CARG#adv. docs=1, k=5, Generator=Llama, Retriever=E52026.02 | 54 | 47 | |
| SDAG-D&A#adv. docs=2, k=5, Generator=Llama, Retriever=E52026.02 | 54 | 36 | |
| top-k decodingModel=Phi-3-medium-4k-instruct, Temp=1, lambda (λ)=0.01, alpha (α)=22025.05 | 52.37 | — | |
| Bregman primal decodingModel=Phi-3-medium-4k-instruct, Temp=1, lambda (λ)=0.01, alpha (α)=1.52025.05 | 52.09 | — | |
| Bregman primal decodingModel=Phi-3-medium-4k-instruct, Temp=1, lambda (λ)=0.01, alpha (α)=22025.05 | 51.75 | — | |
| D&A#adv. docs=2, k=5, Generator=Llama, Retriever=E52026.02 | 51 | 52 | |
| top-k decodingModel=Phi-3-medium-4k-instruct, Temp=1, lambda (λ)=0.01, alpha (α)=1.52025.05 | 50.31 | — | |
| Bregman primal decodingModel=Phi-3-medium-4k-instruct, Temp=1.5, lambda (λ)=0.01, alpha (α)=22025.05 | 50.19 | — | |
| top-k decodingModel=Phi-3-medium-4k-instruct, Temp=1.5, lambda (λ)=0.1, alpha (α)=1.52025.05 | 50.14 | — | |
| Bregman primal decodingModel=Phi-3-medium-4k-instruct, Temp=1.5, lambda (λ)=0.01, alpha (α)=1.52025.05 | 49.47 | — | |
| RAGD#adv. docs=1, k=5, Generator=Llama, Retriever=E52026.02 | 49 | 43 | |
| SDAG-D&A#adv. docs=3, k=10, Generator=Llama, Retriever=E52026.02 | 49 | 40 | |
| CARG#adv. docs=2, k=10, Generator=Llama, Retriever=E52026.02 | 47 | 55 | |
| D&A#adv. docs=3, k=10, Generator=Llama, Retriever=E52026.02 | 47 | 55 | |
| top-k decodingModel=Phi-3-medium-4k-instruct, Temp=1.5, lambda (λ)=0.01, alpha (α)=22025.05 | 45.29 | — | |
| SDAG#adv. docs=2, k=5, Generator=Llama, Retriever=E52026.02 | 45 | 46 | |
| top-k decodingModel=Phi-3-medium-4k-instruct, Temp=1.5, lambda (λ)=0.01, alpha (α)=1.52025.05 | 43.57 | — | |
| CARG#adv. docs=2, k=5, Generator=Llama, Retriever=E52026.02 | 43 | 62 | |
| SDAG#adv. docs=3, k=10, Generator=Llama, Retriever=E52026.02 | 41 | 49 | |
| CARG#adv. docs=3, k=10, Generator=Llama, Retriever=E52026.02 | 37 | 66 |