# Question Answering on TruthfulQA

Source: [Question Answering on TruthfulQA](<https://www.sota2.com/research/sota/question-answering-on-truthfulqa>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-29T07:14:14.082Z

Evaluation of model performance on the TruthfulQA dataset, focusing on truthfulness and accuracy metrics (MC1, MC2).

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Question Answering |
| Dataset (including any stated split) | TruthfulQA |
| Metric | Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 207 of 207 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Accuracy](<https://www.sota2.com/research/sota/question-answering-on-truthfulqa.md?metric=Accuracy>): Higher is better; unit: PERCENT.

- [MC1 Accuracy](<https://www.sota2.com/research/sota/question-answering-on-truthfulqa.md?metric=MC1%20Accuracy>): Higher is better; unit: PERCENT.

- [MC2 Accuracy](<https://www.sota2.com/research/sota/question-answering-on-truthfulqa.md?metric=MC2%20Accuracy>): Higher is better; unit: PERCENT.

- [Overall Score](<https://www.sota2.com/research/sota/question-answering-on-truthfulqa.md?metric=Overall%20Score>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| LLaMA-3.1-8B | 86.6 | Not recorded | 68fa78851e209131cea78ce2 |
| GPT-4o | 85.4 | Not recorded | 68fa78851e209131cea78ce2 |
| SERC | 85 | Not recorded | 6a191d60dbd7150506f82630 |
| GPT-4o | 84.8 | Not recorded | 68fa78851e209131cea78ce2 |
| Re-Ex | 84.5 | Not recorded | 6a191d60dbd7150506f82630 |
| LLaMA-3.1-8B | 82.5 | Not recorded | 68fa78851e209131cea78ce2 |
| CoVe+RAG | 82.5 | Not recorded | 6a191d60dbd7150506f82630 |
| Qwen 2.5-7B | 81.2 | Not recorded | 68fa78851e209131cea78ce2 |
| PSFT | 80.19 | Not recorded | 68fa7ae21e209131cea85a73 |
| SERC | 80 | Not recorded | 6a191d60dbd7150506f82630 |
| LLaMA-3.1-8B | 79.8 | Not recorded | 68fa78851e209131cea78ce2 |
| LLaMA-3.1-70B | 79.3 | Not recorded | 68fa78851e209131cea78ce2 |
| LLaMA-2-7B | 78.6 | Not recorded | 68fa78851e209131cea78ce2 |
| SFT | 77.89 | Not recorded | 68fa7ae21e209131cea85a73 |
| Re-Ex | 77.5 | Not recorded | 6a191d60dbd7150506f82630 |
| LLaMA-2-7B | 77 | Not recorded | 68fa78851e209131cea78ce2 |
| LLaMA-2-7B | 76.7 | Not recorded | 68fa78851e209131cea78ce2 |
| LLaMA-3.1-70B | 76.2 | Not recorded | 68fa78851e209131cea78ce2 |
| Qwen 2.5-7B | 76.2 | Not recorded | 68fa78851e209131cea78ce2 |
| CoVe+RAG | 75 | Not recorded | 6a191d60dbd7150506f82630 |
| Qwen 2.5-7B | 72.6 | Not recorded | 68fa78851e209131cea78ce2 |
| Qwen 2.5-7B | 72.5 | Not recorded | 68fa78851e209131cea78ce2 |
| RARR | 70.5 | Not recorded | 6a191d60dbd7150506f82630 |
| RARR | 68.5 | Not recorded | 6a191d60dbd7150506f82630 |
| Initial Answer | 67.5 | Not recorded | 6a191d60dbd7150506f82630 |
| MAT-STEER | 61.94 | Not recorded | 68fa79e61e209131cea7fdaf |
| Full Fine-tuning | 61.93 | Not recorded | 68fa75381e209131cea66884 |
| LoRA Tuning | 61.93 | Not recorded | 68fa75381e209131cea66884 |
| LLaMA-3.1-8B | 61.6 | Not recorded | 68fa78851e209131cea78ce2 |
| Ours | 61.56 | Not recorded | 68fa75381e209131cea66884 |
| Proxy Tuning | 61.02 | Not recorded | 68fa75381e209131cea66884 |
| Full Fine-tuning | 60.02 | Not recorded | 68fa75381e209131cea66884 |
| CoVe | 60 | Not recorded | 6a191d60dbd7150506f82630 |
| LinUCB+KL | 59.5 | Not recorded | 699e820b6017c4460ffacf00 |
| LITO | 58.63 | Not recorded | 68fa79e61e209131cea7fdaf |
| Forgetting | 58.39 | Not recorded | 68fa7ab91e209131cea84d3f |
| Qwen 2.5-7B | 56.7 | Not recorded | 68fa78851e209131cea78ce2 |
| NL-ITI | 56.67 | Not recorded | 68fa79e61e209131cea7fdaf |
| DPO | 56.1 | Not recorded | 68fa79e61e209131cea7fdaf |
| ICL | 55.32 | Not recorded | 68fa79e61e209131cea7fdaf |
| ICV | 55.21 | Not recorded | 68fa79e61e209131cea7fdaf |
| RAdapt | 55.09 | Not recorded | 68fa79e61e209131cea7fdaf |
| SFT | 54.02 | Not recorded | 68fa79e61e209131cea7fdaf |
| Merge | 53.26 | Not recorded | 68fa79e61e209131cea7fdaf |
| Forgetting | 52.82 | Not recorded | 68fa7ab91e209131cea84d3f |
| ITI | 52.68 | Not recorded | 68fa79e61e209131cea7fdaf |
| Ignoring | 52.38 | Not recorded | 68fa7ab91e209131cea84d3f |
| LLaMA-3.1-8B | 50.6 | Not recorded | 68fa78851e209131cea78ce2 |
| Forgetting | 50.32 | Not recorded | 68fa7ab91e209131cea84d3f |
| Initial Answer | 50 | Not recorded | 6a191d60dbd7150506f82630 |
| Llama-3.1-8B | 49.91 | Not recorded | 68fa79e61e209131cea7fdaf |
| No-Rewrite | 49.6 | Not recorded | 699e820b6017c4460ffacf00 |
| LLAMA PRO INSTRUCT | 48.8 | Not recorded | 68fa736b1e209131cea5c65b |
| LaVIN | 47.9 | Not recorded | 68fa70f81e209131cea4e731 |
| Forgetting | 47.83 | Not recorded | 68fa7ab91e209131cea84d3f |
| HaluSearch (MCTS) | 47.5 | Not recorded | 68fa76771e209131cea6d2be |
| CoVe | 47.5 | Not recorded | 6a191d60dbd7150506f82630 |
| Ignoring | 47.23 | Not recorded | 68fa7ab91e209131cea84d3f |
| Ignoring | 47.05 | Not recorded | 68fa7ab91e209131cea84d3f |
| Base Model | 46.13 | Not recorded | 68fa75381e209131cea66884 |
| LLaMA2-Chat | 45.57 | Not recorded | 68fa736b1e209131cea5c65b |
| Base | 45.08 | Not recorded | 68fa7ab91e209131cea84d3f |
| HaluSearch (MCTS) | 45.07 | Not recorded | 68fa76771e209131cea6d2be |
| Forgetting | 44.83 | Not recorded | 68fa7ab91e209131cea84d3f |
| LLaMA2-7B | 44.6 | Not recorded | 68fa74011e209131cea5f2be |
| Full Tokens | 44.51 | Not recorded | 68fa7ab91e209131cea84d3f |
| WizardMath | 43.65 | Not recorded | 68fa736b1e209131cea5c65b |
| BoN | 43.5 | Not recorded | 68fa76771e209131cea6d2be |
| Ignoring | 43.01 | Not recorded | 68fa7ab91e209131cea84d3f |
| Full Tokens | 42.95 | Not recorded | 68fa7ab91e209131cea84d3f |
| Full Tokens (standard SFT) | 42.65 | Not recorded | 68fa7ab91e209131cea84d3f |
| Ignoring | 42.4 | Not recorded | 68fa7ab91e209131cea84d3f |
| StarCoder | 41.28 | Not recorded | 68fa736b1e209131cea5c65b |
| CodeLLaMA-Instruct | 41.25 | Not recorded | 68fa736b1e209131cea5c65b |
| Forgetting | 40.95 | Not recorded | 68fa7ab91e209131cea84d3f |
| AdamW | 40.8 | Not recorded | 69d48fdb7d0172140ec746df |
| Ignoring | 40.58 | Not recorded | 68fa7ab91e209131cea84d3f |
| 50% KV Pred | 40.02 | Not recorded | 69b8e0070636e5bf4afbe9ae |
| TransAct | 39.6 | Not recorded | 68fa74011e209131cea5f2be |
| Ignoring | 39.56 | Not recorded | 68fa7ab91e209131cea84d3f |

### LLaMA-3.1-8B

Accuracy: 86.6. Result ID: 69958f343badfaffe6892fbb.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Implementation](<https://github.com/MingyuJ666/Disentangling-Memory-and-Reasoning>)

Recorded evaluation configuration: {"Approach":"Ours","Learning Paradigm":"Fine-tuned"}. This record does not assert independent reproduction.

### GPT-4o

Accuracy: 85.4. Result ID: 69958f343badfaffe6892fb3.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa727a1e209131cea56a1e.md>)

[Implementation](<https://github.com/openai/evals>)

Recorded evaluation configuration: {"Approach":"CoT","Learning Paradigm":"Vanilla"}. This record does not assert independent reproduction.

### SERC

Accuracy: 85. Result ID: 6a193cbdfbc373592c131f73.

[Reporting paper](<https://www.sota2.com/research/paper/6a191d60dbd7150506f82630.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a191d60dbd7150506f82630.md>)

Recorded evaluation configuration: {"Model":"14B"}. This record does not assert independent reproduction.

### GPT-4o

Accuracy: 84.8. Result ID: 69958f343badfaffe6892fae.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa727a1e209131cea56a1e.md>)

[Implementation](<https://github.com/openai/evals>)

Recorded evaluation configuration: {"Approach":"Zero-shot","Learning Paradigm":"Vanilla"}. This record does not assert independent reproduction.

### Re-Ex

Accuracy: 84.5. Result ID: 6a193cbdfbc373592c131f72.

[Reporting paper](<https://www.sota2.com/research/paper/6a191d60dbd7150506f82630.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77ea1e209131cea75f4b.md>)

Recorded evaluation configuration: {"Model":"14B"}. This record does not assert independent reproduction.

### LLaMA-3.1-8B

Accuracy: 82.5. Result ID: 69958f343badfaffe6892fb8.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Implementation](<https://github.com/MingyuJ666/Disentangling-Memory-and-Reasoning>)

Recorded evaluation configuration: {"Approach":"Planning-token","Learning Paradigm":"Fine-tuned"}. This record does not assert independent reproduction.

### CoVe+RAG

Accuracy: 82.5. Result ID: 6a193cbdfbc373592c131f70.

[Reporting paper](<https://www.sota2.com/research/paper/6a191d60dbd7150506f82630.md>)

Recorded evaluation configuration: {"Model":"14B"}. This record does not assert independent reproduction.

### Qwen 2.5-7B

Accuracy: 81.2. Result ID: 69958f343badfaffe6892fbc.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Implementation](<https://github.com/MingyuJ666/Disentangling-Memory-and-Reasoning>)

Recorded evaluation configuration: {"Approach":"Ours","Learning Paradigm":"Fine-tuned"}. This record does not assert independent reproduction.

### PSFT

Accuracy: 80.19. Result ID: 69ddf9b647f34ce924c55c0d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ae21e209131cea85a73.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7ae21e209131cea85a73.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### SERC

Accuracy: 80. Result ID: 6a193cbdfbc373592c131f6d.

[Reporting paper](<https://www.sota2.com/research/paper/6a191d60dbd7150506f82630.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a191d60dbd7150506f82630.md>)

Recorded evaluation configuration: {"Model":"8B"}. This record does not assert independent reproduction.

### LLaMA-3.1-8B

Accuracy: 79.8. Result ID: 69958f343badfaffe6892fb5.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Implementation](<https://github.com/MingyuJ666/Disentangling-Memory-and-Reasoning>)

Recorded evaluation configuration: {"Approach":"LoRA","Learning Paradigm":"Fine-tuned"}. This record does not assert independent reproduction.

### LLaMA-3.1-70B

Accuracy: 79.3. Result ID: 69958f343badfaffe6892fac.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75d11e209131cea6a028.md>)

[Implementation](<https://github.com/meta-llama/llama-agentic-system>)

Recorded evaluation configuration: {"Approach":"Zero-shot","Learning Paradigm":"Vanilla"}. This record does not assert independent reproduction.

### LLaMA-2-7B

Accuracy: 78.6. Result ID: 69958f343badfaffe6892fba.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Implementation](<https://github.com/MingyuJ666/Disentangling-Memory-and-Reasoning>)

Recorded evaluation configuration: {"Approach":"Ours","Learning Paradigm":"Fine-tuned"}. This record does not assert independent reproduction.

### SFT

Accuracy: 77.89. Result ID: 69ddf9b647f34ce924c55c0e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ae21e209131cea85a73.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77dd1e209131cea75a95.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### Re-Ex

Accuracy: 77.5. Result ID: 6a193cbdfbc373592c131f6c.

[Reporting paper](<https://www.sota2.com/research/paper/6a191d60dbd7150506f82630.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77ea1e209131cea75f4b.md>)

Recorded evaluation configuration: {"Model":"8B"}. This record does not assert independent reproduction.

### LLaMA-2-7B

Accuracy: 77. Result ID: 69958f343badfaffe6892fb7.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Implementation](<https://github.com/MingyuJ666/Disentangling-Memory-and-Reasoning>)

Recorded evaluation configuration: {"Approach":"Planning-token","Learning Paradigm":"Fine-tuned"}. This record does not assert independent reproduction.

### LLaMA-2-7B

Accuracy: 76.7. Result ID: 69958f343badfaffe6892fb4.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Implementation](<https://github.com/MingyuJ666/Disentangling-Memory-and-Reasoning>)

Recorded evaluation configuration: {"Approach":"LoRA","Learning Paradigm":"Fine-tuned"}. This record does not assert independent reproduction.

### LLaMA-3.1-70B

Accuracy: 76.2. Result ID: 69958f343badfaffe6892fb1.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75d11e209131cea6a028.md>)

[Implementation](<https://github.com/meta-llama/llama-agentic-system>)

Recorded evaluation configuration: {"Approach":"CoT","Learning Paradigm":"Vanilla"}. This record does not assert independent reproduction.

### Qwen 2.5-7B

Accuracy: 76.2. Result ID: 69958f343badfaffe6892fb9.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Implementation](<https://github.com/MingyuJ666/Disentangling-Memory-and-Reasoning>)

Recorded evaluation configuration: {"Approach":"Planning-token","Learning Paradigm":"Fine-tuned"}. This record does not assert independent reproduction.

### CoVe+RAG

Accuracy: 75. Result ID: 6a193cbdfbc373592c131f6a.

[Reporting paper](<https://www.sota2.com/research/paper/6a191d60dbd7150506f82630.md>)

Recorded evaluation configuration: {"Model":"8B"}. This record does not assert independent reproduction.

### Qwen 2.5-7B

Accuracy: 72.6. Result ID: 69958f343badfaffe6892fad.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Implementation](<https://github.com/MingyuJ666/Disentangling-Memory-and-Reasoning>)

Recorded evaluation configuration: {"Approach":"Zero-shot","Learning Paradigm":"Vanilla"}. This record does not assert independent reproduction.

### Qwen 2.5-7B

Accuracy: 72.5. Result ID: 69958f343badfaffe6892fb6.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Implementation](<https://github.com/MingyuJ666/Disentangling-Memory-and-Reasoning>)

Recorded evaluation configuration: {"Approach":"LoRA","Learning Paradigm":"Fine-tuned"}. This record does not assert independent reproduction.

### RARR

Accuracy: 70.5. Result ID: 6a193cbdfbc373592c131f71.

[Reporting paper](<https://www.sota2.com/research/paper/6a191d60dbd7150506f82630.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f961e209131cea4676a.md>)

[Implementation](<https://github.com/anthonywchen/RARR>)

Recorded evaluation configuration: {"Model":"14B"}. This record does not assert independent reproduction.

### RARR

Accuracy: 68.5. Result ID: 6a193cbdfbc373592c131f6b.

[Reporting paper](<https://www.sota2.com/research/paper/6a191d60dbd7150506f82630.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f961e209131cea4676a.md>)

[Implementation](<https://github.com/anthonywchen/RARR>)

Recorded evaluation configuration: {"Model":"8B"}. This record does not assert independent reproduction.

### Initial Answer

Accuracy: 67.5. Result ID: 6a193cbdfbc373592c131f6e.

[Reporting paper](<https://www.sota2.com/research/paper/6a191d60dbd7150506f82630.md>)

Recorded evaluation configuration: {"Model":"14B"}. This record does not assert independent reproduction.

### MAT-STEER

Accuracy: 61.94. Result ID: 698c551437894ef7d9cd950f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79e61e209131cea7fdaf.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79e61e209131cea7fdaf.md>)

[Implementation](<https://github.com/duykhuongnguyen/MAT-Steer>)

Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B"}. This record does not assert independent reproduction.

### Full Fine-tuning

Accuracy: 61.93. Result ID: 699aaf395698665ec43c7faf.

[Reporting paper](<https://www.sota2.com/research/paper/68fa75381e209131cea66884.md>)

Recorded evaluation configuration: {"Model Size":"13B","Transfer Source":"N/A"}. This record does not assert independent reproduction.

### LoRA Tuning

Accuracy: 61.93. Result ID: 699aaf395698665ec43c7fb0.

[Reporting paper](<https://www.sota2.com/research/paper/68fa75381e209131cea66884.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6b121e209131cea2bf59.md>)

[Implementation](<https://github.com/microsoft/LoRA>)

Recorded evaluation configuration: {"Model Size":"13B","Transfer Source":"N/A"}. This record does not assert independent reproduction.

### LLaMA-3.1-8B

Accuracy: 61.6. Result ID: 69958f343badfaffe6892fab.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Implementation](<https://github.com/MingyuJ666/Disentangling-Memory-and-Reasoning>)

Recorded evaluation configuration: {"Approach":"Zero-shot","Learning Paradigm":"Vanilla"}. This record does not assert independent reproduction.

### Ours

Accuracy: 61.56. Result ID: 699aaf395698665ec43c7fb6.

[Reporting paper](<https://www.sota2.com/research/paper/68fa75381e209131cea66884.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75381e209131cea66884.md>)

[Implementation](<https://github.com/Facico/Dynamic-Logit-Fusion>)

Recorded evaluation configuration: {"Model Size":"13B","Transfer Source":"7B"}. This record does not assert independent reproduction.

### Proxy Tuning

Accuracy: 61.02. Result ID: 699aaf395698665ec43c7fb5.

[Reporting paper](<https://www.sota2.com/research/paper/68fa75381e209131cea66884.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75381e209131cea66884.md>)

[Implementation](<https://github.com/Facico/Dynamic-Logit-Fusion>)

Recorded evaluation configuration: {"Model Size":"13B","Transfer Source":"7B"}. This record does not assert independent reproduction.

### Full Fine-tuning

Accuracy: 60.02. Result ID: 699aaf395698665ec43c7fb4.

[Reporting paper](<https://www.sota2.com/research/paper/68fa75381e209131cea66884.md>)

Recorded evaluation configuration: {"Model Size":"13B","Transfer Source":"7B"}. This record does not assert independent reproduction.

### CoVe

Accuracy: 60. Result ID: 6a193cbdfbc373592c131f6f.

[Reporting paper](<https://www.sota2.com/research/paper/6a191d60dbd7150506f82630.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70b41e209131cea4cb37.md>)

Recorded evaluation configuration: {"Model":"14B"}. This record does not assert independent reproduction.

### LinUCB+KL

Accuracy: 59.5. Result ID: 699eaa375698665ec4766fa1.

[Reporting paper](<https://www.sota2.com/research/paper/699e820b6017c4460ffacf00.md>)

[Method origin paper](<https://www.sota2.com/research/paper/699e820b6017c4460ffacf00.md>)

Recorded evaluation configuration: {"Algorithm Family":"Contextual Linear"}. This record does not assert independent reproduction.

### LITO

Accuracy: 58.63. Result ID: 698c551437894ef7d9cd950e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79e61e209131cea7fdaf.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa738f1e209131cea5d3cb.md>)

Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B"}. This record does not assert independent reproduction.

### Forgetting

Accuracy: 58.39. Result ID: 69cb741447f34ce924b35042.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ab91e209131cea84d3f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7ab91e209131cea84d3f.md>)

Recorded evaluation configuration: {"Base model":"LLaMA-3.1-8B","Granularity":"token-level"}. This record does not assert independent reproduction.

### Qwen 2.5-7B

Accuracy: 56.7. Result ID: 69958f343badfaffe6892fb2.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Implementation](<https://github.com/MingyuJ666/Disentangling-Memory-and-Reasoning>)

Recorded evaluation configuration: {"Approach":"CoT","Learning Paradigm":"Vanilla"}. This record does not assert independent reproduction.

### NL-ITI

Accuracy: 56.67. Result ID: 698c551437894ef7d9cd950d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79e61e209131cea7fdaf.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa738b1e209131cea5d1df.md>)

Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B"}. This record does not assert independent reproduction.

### DPO

Accuracy: 56.1. Result ID: 698c551437894ef7d9cd9508.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79e61e209131cea7fdaf.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74511e209131cea6105d.md>)

Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B","Protocol":"Direct Preference Optimization"}. This record does not assert independent reproduction.

### ICL

Accuracy: 55.32. Result ID: 698c551437894ef7d9cd9506.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79e61e209131cea7fdaf.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa680c1e209131cea19d53.md>)

[Implementation](<https://github.com/openai/gpt-3/blob/master/overlap_frequency.md>)

Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B","Protocol":"In-context learning"}. This record does not assert independent reproduction.

### ICV

Accuracy: 55.21. Result ID: 698c551437894ef7d9cd950c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79e61e209131cea7fdaf.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa723d1e209131cea55314.md>)

Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B"}. This record does not assert independent reproduction.

### RAdapt

Accuracy: 55.09. Result ID: 698c551437894ef7d9cd950a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79e61e209131cea7fdaf.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72391e209131cea5515d.md>)

Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B"}. This record does not assert independent reproduction.

### SFT

Accuracy: 54.02. Result ID: 698c551437894ef7d9cd9507.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79e61e209131cea7fdaf.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6bf71e209131cea314c0.md>)

[Implementation](<https://github.com/openai/following-instructions-human-feedback>)

Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B","Protocol":"Supervised Fine-Tuning"}. This record does not assert independent reproduction.

### Merge

Accuracy: 53.26. Result ID: 698c551437894ef7d9cd9509.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79e61e209131cea7fdaf.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6ce11e209131cea3679c.md>)

[Implementation](<https://github.com/mlfoundations/model-soups>)

Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B","Protocol":"Model Merging"}. This record does not assert independent reproduction.

### Forgetting

Accuracy: 52.82. Result ID: 69cb741447f34ce924b3506a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ab91e209131cea84d3f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7ab91e209131cea84d3f.md>)

Recorded evaluation configuration: {"Base model":"LLaMA-2-13B"}. This record does not assert independent reproduction.

### ITI

Accuracy: 52.68. Result ID: 698c551437894ef7d9cd950b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79e61e209131cea7fdaf.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73e31e209131cea5eb16.md>)

Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B"}. This record does not assert independent reproduction.

### Ignoring

Accuracy: 52.38. Result ID: 69cb741447f34ce924b35041.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ab91e209131cea84d3f.md>)

Recorded evaluation configuration: {"Base model":"LLaMA-3.1-8B","Granularity":"token-level"}. This record does not assert independent reproduction.

### LLaMA-3.1-8B

Accuracy: 50.6. Result ID: 69958f343badfaffe6892fb0.

[Reporting paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78851e209131cea78ce2.md>)

[Implementation](<https://github.com/MingyuJ666/Disentangling-Memory-and-Reasoning>)

Recorded evaluation configuration: {"Approach":"CoT","Learning Paradigm":"Vanilla"}. This record does not assert independent reproduction.

### Forgetting

Accuracy: 50.32. Result ID: 69cb741447f34ce924b3503c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ab91e209131cea84d3f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7ab91e209131cea84d3f.md>)

Recorded evaluation configuration: {"Base model":"LLaMA-3.2-3B","Granularity":"token-level"}. This record does not assert independent reproduction.

### Initial Answer

Accuracy: 50. Result ID: 6a193cbdfbc373592c131f68.

[Reporting paper](<https://www.sota2.com/research/paper/6a191d60dbd7150506f82630.md>)

Recorded evaluation configuration: {"Model":"8B"}. This record does not assert independent reproduction.

### Llama-3.1-8B

Accuracy: 49.91. Result ID: 698c551437894ef7d9cd9505.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79e61e209131cea7fdaf.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75d11e209131cea6a028.md>)

[Implementation](<https://github.com/meta-llama/llama-agentic-system>)

Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B","Protocol":"Base model"}. This record does not assert independent reproduction.

### No-Rewrite

Accuracy: 49.6. Result ID: 699eaa375698665ec4766fa0.

[Reporting paper](<https://www.sota2.com/research/paper/699e820b6017c4460ffacf00.md>)

Recorded evaluation configuration: {"Algorithm Family":"Base"}. This record does not assert independent reproduction.

### LLAMA PRO INSTRUCT

Accuracy: 48.8. Result ID: 6992e9cf3a6c314849125456.

[Reporting paper](<https://www.sota2.com/research/paper/68fa736b1e209131cea5c65b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa736b1e209131cea5c65b.md>)

[Implementation](<https://github.com/TencentARC/LLaMA-Pro>)

Recorded evaluation configuration: {"Training Stage":"SFT comparison"}. This record does not assert independent reproduction.

### LaVIN

Accuracy: 47.9. Result ID: 699169273a6c31484907f5d4.

[Reporting paper](<https://www.sota2.com/research/paper/68fa70f81e209131cea4e731.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70f81e209131cea4e731.md>)

[Implementation](<https://luogen1996.github.io/lavin>)

Recorded evaluation configuration: {"zero-shot":true,"setup":"Mc1\_targets"}. This record does not assert independent reproduction.

### Forgetting

Accuracy: 47.83. Result ID: 69cb741447f34ce924b35040.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ab91e209131cea84d3f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7ab91e209131cea84d3f.md>)

Recorded evaluation configuration: {"Base model":"LLaMA-3.1-8B","Granularity":"sequence-level"}. This record does not assert independent reproduction.

### HaluSearch (MCTS)

Accuracy: 47.5. Result ID: 69942e803a6c3148491ba234.

[Reporting paper](<https://www.sota2.com/research/paper/68fa76771e209131cea6d2be.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76771e209131cea6d2be.md>)

Recorded evaluation configuration: {"Backbone":"Llama3.1-8B-Instruct"}. This record does not assert independent reproduction.

### CoVe

Accuracy: 47.5. Result ID: 6a193cbdfbc373592c131f69.

[Reporting paper](<https://www.sota2.com/research/paper/6a191d60dbd7150506f82630.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70b41e209131cea4cb37.md>)

Recorded evaluation configuration: {"Model":"8B"}. This record does not assert independent reproduction.

### Ignoring

Accuracy: 47.23. Result ID: 69cb741447f34ce924b3503b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ab91e209131cea84d3f.md>)

Recorded evaluation configuration: {"Base model":"LLaMA-3.2-3B","Granularity":"token-level"}. This record does not assert independent reproduction.

### Ignoring

Accuracy: 47.05. Result ID: 69cb741447f34ce924b3503f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ab91e209131cea84d3f.md>)

Recorded evaluation configuration: {"Base model":"LLaMA-3.1-8B","Granularity":"sequence-level"}. This record does not assert independent reproduction.

### Base Model

Accuracy: 46.13. Result ID: 699aaf395698665ec43c7fae.

[Reporting paper](<https://www.sota2.com/research/paper/68fa75381e209131cea66884.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa701f1e209131cea492fe.md>)

[Implementation](<https://github.com/facebookresearch/llama>)

Recorded evaluation configuration: {"Model Size":"13B","Transfer Source":"N/A"}. This record does not assert independent reproduction.

### LLaMA2-Chat

Accuracy: 45.57. Result ID: 6992e9cf3a6c314849125457.

[Reporting paper](<https://www.sota2.com/research/paper/68fa736b1e209131cea5c65b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa701f1e209131cea492fe.md>)

[Implementation](<https://github.com/facebookresearch/llama>)

Recorded evaluation configuration: {"Parameters":"7B","Training Stage":"SFT comparison"}. This record does not assert independent reproduction.

### Base

Accuracy: 45.08. Result ID: 69cb741447f34ce924b3503d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ab91e209131cea84d3f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75d11e209131cea6a028.md>)

[Implementation](<https://github.com/meta-llama/llama-agentic-system>)

Recorded evaluation configuration: {"Base model":"LLaMA-3.1-8B"}. This record does not assert independent reproduction.

### HaluSearch (MCTS)

Accuracy: 45.07. Result ID: 69942e803a6c3148491ba23b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa76771e209131cea6d2be.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76771e209131cea6d2be.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2-7B-Instruct"}. This record does not assert independent reproduction.

### Forgetting

Accuracy: 44.83. Result ID: 69cb741447f34ce924b35036.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ab91e209131cea84d3f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7ab91e209131cea84d3f.md>)

Recorded evaluation configuration: {"Base model":"LLaMA-3.2-1B","Granularity":"token-level"}. This record does not assert independent reproduction.

### LLaMA2-7B

Accuracy: 44.6. Result ID: 699329653a6c31484914255a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa74011e209131cea5f2be.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa701f1e209131cea492fe.md>)

[Implementation](<https://github.com/facebookresearch/llama>)

Recorded evaluation configuration: {"Number of Parameters":"7B","Backbone":"LLaMA2","Shots":0,"Prepended Examples":6}. This record does not assert independent reproduction.

### Full Tokens

Accuracy: 44.51. Result ID: 69cb741447f34ce924b3503e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ab91e209131cea84d3f.md>)

Recorded evaluation configuration: {"Base model":"LLaMA-3.1-8B","Training Protocol":"standard SFT"}. This record does not assert independent reproduction.

### WizardMath

Accuracy: 43.65. Result ID: 6992e9cf3a6c31484912545a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa736b1e209131cea5c65b.md>)

Recorded evaluation configuration: {"Parameters":"7B","Training Stage":"SFT comparison"}. This record does not assert independent reproduction.

### BoN

Accuracy: 43.5. Result ID: 69942e803a6c3148491ba231.

[Reporting paper](<https://www.sota2.com/research/paper/68fa76771e209131cea6d2be.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f9a1e209131cea468c7.md>)

[Implementation](<https://github.com/openai/prm800k>)

Recorded evaluation configuration: {"Backbone":"Llama3.1-8B-Instruct"}. This record does not assert independent reproduction.

### Ignoring

Accuracy: 43.01. Result ID: 69cb741447f34ce924b35069.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ab91e209131cea84d3f.md>)

Recorded evaluation configuration: {"Base model":"LLaMA-2-13B"}. This record does not assert independent reproduction.

### Full Tokens

Accuracy: 42.95. Result ID: 69cb741447f34ce924b35038.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ab91e209131cea84d3f.md>)

Recorded evaluation configuration: {"Base model":"LLaMA-3.2-3B","Training Protocol":"standard SFT"}. This record does not assert independent reproduction.

### Full Tokens (standard SFT)

Accuracy: 42.65. Result ID: 69cb741447f34ce924b35068.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ab91e209131cea84d3f.md>)

Recorded evaluation configuration: {"Base model":"LLaMA-2-13B"}. This record does not assert independent reproduction.

### Ignoring

Accuracy: 42.4. Result ID: 69cb741447f34ce924b35035.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ab91e209131cea84d3f.md>)

Recorded evaluation configuration: {"Base model":"LLaMA-3.2-1B","Granularity":"token-level"}. This record does not assert independent reproduction.

### StarCoder

Accuracy: 41.28. Result ID: 6992e9cf3a6c314849125452.

[Reporting paper](<https://www.sota2.com/research/paper/68fa736b1e209131cea5c65b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa717d1e209131cea5197b.md>)

Recorded evaluation configuration: {"Parameters":"15B","Training Stage":"Pretrained comparison"}. This record does not assert independent reproduction.

### CodeLLaMA-Instruct

Accuracy: 41.25. Result ID: 6992e9cf3a6c314849125458.

[Reporting paper](<https://www.sota2.com/research/paper/68fa736b1e209131cea5c65b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71ff1e209131cea53ff7.md>)

[Implementation](<https://github.com/facebookresearch/codellama>)

Recorded evaluation configuration: {"Parameters":"7B","Training Stage":"SFT comparison"}. This record does not assert independent reproduction.

### Forgetting

Accuracy: 40.95. Result ID: 69cb741447f34ce924b3503a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ab91e209131cea84d3f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7ab91e209131cea84d3f.md>)

Recorded evaluation configuration: {"Base model":"LLaMA-3.2-3B","Granularity":"sequence-level"}. This record does not assert independent reproduction.

### AdamW

Accuracy: 40.8. Result ID: 69d4d71d47f34ce924bd2532.

[Reporting paper](<https://www.sota2.com/research/paper/69d48fdb7d0172140ec746df.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa55761e209131ce9946db.md>)

Recorded evaluation configuration: {"Runtime":"8.8"}. This record does not assert independent reproduction.

### Ignoring

Accuracy: 40.58. Result ID: 69cb741447f34ce924b35039.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ab91e209131cea84d3f.md>)

Recorded evaluation configuration: {"Base model":"LLaMA-3.2-3B","Granularity":"sequence-level"}. This record does not assert independent reproduction.

### 50% KV Pred

Accuracy: 40.02. Result ID: 69b9037247f34ce924a129d1.

[Reporting paper](<https://www.sota2.com/research/paper/69b8e0070636e5bf4afbe9ae.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b8e0070636e5bf4afbe9ae.md>)

Recorded evaluation configuration: {"Model":"Qwen3 32B"}. This record does not assert independent reproduction.

### TransAct

Accuracy: 39.6. Result ID: 699329653a6c314849142562.

[Reporting paper](<https://www.sota2.com/research/paper/68fa74011e209131cea5f2be.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74011e209131cea5f2be.md>)

[Implementation](<https://github.com/sbwww/TransAct-pruning>)

Recorded evaluation configuration: {"Number of Parameters":"1.3B","Backbone":"LLaMA","Shots":0,"Prepended Examples":6}. This record does not assert independent reproduction.

### Ignoring

Accuracy: 39.56. Result ID: 69cb741447f34ce924b35033.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7ab91e209131cea84d3f.md>)

Recorded evaluation configuration: {"Base model":"LLaMA-3.2-1B","Granularity":"sequence-level"}. This record does not assert independent reproduction.

Page 1 of 3. This is an index page, not the full collection.

- [Next index page](<https://www.sota2.com/research/sota/question-answering-on-truthfulqa.md?metric=Accuracy&page=2>): Follow this link to discover more pages.
