Structural Consistency on Consistency Evaluation Dataset (N=720) (test)
1Structural Consistency ScoreClaude-3.7-Sonnet
Evaluation Results
| Method | Links | |
|---|---|---|
| Claude-3.7-SonnetTemperature=0.12025.11 | 1 | |
| Claude-3.7-SonnetTemperature=0.22025.11 | 1 | |
| Claude-3.7-SonnetTemperature=0.32025.11 | 1 | |
| Claude-3.7-SonnetTemperature=0.42025.11 | 1 | |
| Claude-3.7-SonnetTemperature=0.52025.11 | 1 | |
| Claude-3.7-SonnetTemperature=0.62025.11 | 1 | |
| Claude-3.7-SonnetTemperature=0.72025.11 | 1 | |
| Claude-3.7-SonnetTemperature=0.82025.11 | 1 | |
| Claude-3.7-SonnetTemperature=0.92025.11 | 0.988 | |
| Claude-3.5-Sonnet-V2Temperature=0.22025.11 | 0.975 | |
| Claude-3.5-Sonnet-V2Temperature=0.32025.11 | 0.963 | |
| Claude-3.5-Sonnet-V2Temperature=0.62025.11 | 0.963 | |
| Claude-3.5-Sonnet-V2Temperature=0.42025.11 | 0.95 | |
| Claude-3.5-Sonnet-V2Temperature=0.52025.11 | 0.95 | |
| Claude-3.5-Sonnet-V2Temperature=0.72025.11 | 0.95 | |
| Claude-3.5-Sonnet-V2Temperature=0.92025.11 | 0.95 | |
| Claude-3.5-Sonnet-V2Temperature=0.82025.11 | 0.938 | |
| Claude-3.5-Sonnet-V2Temperature=0.12025.11 | 0.875 | |
| Claude-3.5-HaikuTemperature=0.22025.11 | 0.869 | |
| Llama-3.3-70BTemperature=0.22025.11 | 0.863 | |
| Claude-3.5-HaikuTemperature=0.12025.11 | 0.86 | |
| Llama-3.3-70BTemperature=0.62025.11 | 0.85 | |
| Llama-3.3-70BTemperature=0.82025.11 | 0.838 | |
| Llama-3.3-70BTemperature=0.92025.11 | 0.838 | |
| Llama-3.3-70BTemperature=0.12025.11 | 0.825 | |
| Llama-3.3-70BTemperature=0.32025.11 | 0.825 | |
| Llama-3.3-70BTemperature=0.72025.11 | 0.825 | |
| Llama-3.3-70BTemperature=0.42025.11 | 0.8 | |
| Llama-3.3-70BTemperature=0.52025.11 | 0.8 | |
| Nova-ProTemperature=0.52025.11 | 0.787 | |
| Claude-3-HaikuTemperature=0.32025.11 | 0.781 | |
| Claude-3.5-HaikuTemperature=0.32025.11 | 0.776 | |
| Nova-ProTemperature=0.22025.11 | 0.775 | |
| Claude-3.5-HaikuTemperature=0.62025.11 | 0.771 | |
| Claude-3.5-HaikuTemperature=0.42025.11 | 0.768 | |
| Claude-3-HaikuTemperature=0.42025.11 | 0.767 | |
| Claude-3-HaikuTemperature=0.52025.11 | 0.758 | |
| Claude-3.5-HaikuTemperature=0.52025.11 | 0.757 | |
| Claude-3-HaikuTemperature=0.12025.11 | 0.755 | |
| Claude-3-HaikuTemperature=0.22025.11 | 0.75 | |
| Nova-ProTemperature=0.62025.11 | 0.749 | |
| Nova-ProTemperature=0.82025.11 | 0.748 | |
| Nova-ProTemperature=0.42025.11 | 0.739 | |
| Claude-3-HaikuTemperature=0.92025.11 | 0.722 | |
| Claude-3.5-HaikuTemperature=0.82025.11 | 0.722 | |
| Nova-ProTemperature=0.12025.11 | 0.722 | |
| Claude-3-HaikuTemperature=0.82025.11 | 0.708 | |
| Claude-3.5-HaikuTemperature=0.92025.11 | 0.704 | |
| Claude-3.5-HaikuTemperature=0.72025.11 | 0.703 | |
| Nova-ProTemperature=0.32025.11 | 0.7 | |
| Claude-3-HaikuTemperature=0.72025.11 | 0.682 | |
| Claude-3-HaikuTemperature=0.62025.11 | 0.679 | |
| Nova-ProTemperature=0.92025.11 | 0.673 | |
| Nova-ProTemperature=0.72025.11 | 0.658 |