PRP Faithfulness Evaluation on Eve 30 persona statements (original characters)
0.7ΔAPC (DeB)Gemma-7B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Gemma-7BCPO (APC-based DPO)=False2024.05 | 0.7 | -0.2 | 2 | |
| Experience Upload (EU)CPO (APC-based DPO)=False2024.05 | 3.6 | 0.7 | 4.6 | |
| Long-context Memory (LCM)CPO (APC-based DPO)=False2024.05 | 3.9 | 0.7 | 5 | |
| Experience Upload (EU)CPO (APC-based DPO)=True2024.05 | 3.9 | 0.9 | 5.2 | |
| Retrieval-augmented Generation (RAG)CPO (APC-based DPO)=False2024.05 | 4.8 | 2.4 | 5.8 | |
| Long-context Memory (LCM)CPO (APC-based DPO)=True2024.05 | 5.1 | 3.3 | 6.6 | |
| Retrieval-augmented Generation (RAG)CPO (APC-based DPO)=True2024.05 | 5.8 | 4.2 | 7 |