PRP Faithfulness Evaluation on Bob original characters 19 persona statements
1.1ΔAPC (DeB)Gemma-7B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Gemma-7BCPO (APC-based DPO)=False2024.05 | 1.1 | 0.4 | 1.8 | |
| Experience Upload (EU)CPO (APC-based DPO)=False2024.05 | 3.4 | 1.1 | 6.2 | |
| Experience Upload (EU)CPO (APC-based DPO)=True2024.05 | 3.8 | 1.8 | 6.8 | |
| Retrieval-augmented Generation (RAG)CPO (APC-based DPO)=False2024.05 | 4 | 1.7 | 6.8 | |
| Long-context Memory (LCM)CPO (APC-based DPO)=False2024.05 | 4.5 | 2.2 | 7.2 | |
| Retrieval-augmented Generation (RAG)CPO (APC-based DPO)=True2024.05 | 5.2 | 3.8 | 8.2 | |
| Long-context Memory (LCM)CPO (APC-based DPO)=True2024.05 | 5.3 | 2.5 | 7.8 | |
| Gemma-7BCPO Status=Without CPO2024.05 | 9.7 | 7.3 | 1.8 | |
| EUCPO Status=Without CPO2024.05 | 12 | 8 | 6.2 | |
| EUCPO Status=With CPO2024.05 | 12.4 | 8.7 | 6.8 | |
| RAGCPO Status=Without CPO2024.05 | 12.6 | 8.6 | 6.8 | |
| LCMCPO Status=Without CPO2024.05 | 13.1 | 9.1 | 7.2 | |
| RAGCPO Status=With CPO2024.05 | 13.8 | 10.7 | 8.2 | |
| LCMCPO Status=With CPO2024.05 | 13.9 | 9.4 | 7.8 |