List your company
ResearchFormerly WizWand

Summarization on ACI-Bench (test)

41ROUGE-L F1CriSPO
24.56828.83433.137.366Oct 3, 2024
Updated 8mo ago

Evaluation results

MethodLinks
CriSPOLLM=Claude3 Sonnet, Shots=32024.1041
ManualLLM=Claude3 Sonnet, Shots=32024.1038.6
CriSPOLLM=Claude Instant, Shots=32024.1035.3
ManualLLM=Claude Instant, Shots=32024.1033.5
CriSPOLLM=Claude3 Sonnet, Shots=02024.1030.3
CriSPOLLM=Llama3 8B, Shots=0, Optimizer=Claude3 Sonnet2024.1029.9
OPROLLM=Claude3 Sonnet, Shots=02024.1029.5
ManualLLM=Claude3 Sonnet, Shots=02024.1029.3
OPROLLM=Llama3 8B, Shots=0, Optimizer=Claude3 Sonnet2024.1029.3
ManualLLM=Mistral 7B, Shots=32024.1028.1
ManualLLM=Llama3 8B, Shots=02024.1027.7
CriSPOLLM=Claude Instant, Shots=02024.1026.8
CriSPOLLM=Mistral 7B, Shots=32024.1026.2
ManualLLM=Claude Instant, Shots=02024.1026.1
CriSPOLLM=Mistral 7B, Shots=02024.1025.6
OPROLLM=Claude Instant, Shots=02024.1025.5
ManualLLM=Mistral 7B, Shots=02024.1025.4
OPROLLM=Mistral 7B, Shots=02024.1025.2