Loading the SOTA2 catalog…
Evaluations
Run LLM-as-a-Judge evaluations on any LLM, dedicated endpoint, or inference routing policy; compare quality and performance using your own datasets.
What is Evaluations?
Run LLM-as-a-Judge evaluations on any LLM, dedicated endpoint, or inference routing policy; compare quality and performance using your own datasets.
Product typeAI Evaluation & Observability
Pricing · Usage Based
Deployment
APINot available
What teams use Evaluations for
AI/ML inference
Application hosting and deployment
Managed databases
Cloud compute
Object storage
Serverless computing
Integrations and access
Platforms, integrations, and language support vary by plan and region. Confirm final requirements with the vendor.
Platforms
IntegrationsAPI and webhooks
Languages
Support
What reviewers say
Community reviewsReviews are owned and editable by their authors.
Sign in to reviewLoading community reviews…
Enterprise readiness
Compliance claims are normalized from current vendor documentation and independently reviewed by SOTA2.
SOC 2
Cloud Firewalls
SSH keys
DdoS mitigation
Kata Containers for App Platform isolation
Encrypted environment variables
Etcd encrypted at rest
Cilium network policies
Standard Kubernetes network policies
Kubernetes Private Distributors List membership
Trusted Sources for Managed Databases
VPC isolation for worker nodes
HTTPS and TLS by default for Functions
Data retention:
Benchmarks and comparisons
Independent benchmarksStructured task results are being verified.
Side-by-side comparisonsComparison workspaces will be available in a later release.
