Translation on Translation misgendering evaluation set into English zero-shot SynthBio v3 (test)
97.2Accuracy (Overall)PaLM 540B
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| PaLM 540Bmode=zero-shot2023.05 | 97.2 | 100 | 94.4 | 86.4 | 91.4 | 50 | |
| PaLM 2 (L)mode=zero-shot, checkpoints=5 checkpoints2023.05 | 97.2 | 99.9 | 94.5 | 87.3 | 89.5 | 58.7 |