Role-Play Faithfulness on PRP - Characters with persona statements at scale (test)
70.1Spartacus (APC)RAG
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| RAGConfiguration=w/o CPO2024.05 | 70.1 | 130 | 174.6 | 328.1 | 425.4 | 477.5 | 267.6 | |
| RAGConfiguration=w/ CPO2024.05 | 69.9 | 130.9 | 176.6 | 327.7 | 426.4 | 481.1 | 268.8 | |
| Gemma-7BConfiguration=w/ CPO2024.05 | 69.4 | 130.2 | 176.4 | 326.7 | 423.1 | 479.3 | 267.5 | |
| ChatGPTConfiguration=GPT baseline2024.05 | 69.1 | 128.4 | 168.6 | 324.1 | 421.7 | 473.1 | 264.2 | |
| GPT-4Configuration=GPT baseline2024.05 | 69 | 129.5 | 169.6 | 326.2 | 425.1 | 478.3 | 266 | |
| Gemma-7BConfiguration=w/o CPO2024.05 | 68.8 | 129.3 | 173 | 327.1 | 423.5 | 470.8 | 265.4 | |
| EUConfiguration=w/ CPO2024.05 | 68.7 | 127.8 | 170.9 | 324.5 | 418.7 | 468.1 | 263.1 | |
| EUConfiguration=w/o CPO2024.05 | 67.3 | 125.9 | 166.1 | 321.5 | 418.4 | 468 | 261.2 |