List your company

October 19, 2026 edition

Best AI to evaluate AI systems in technology and software

Test model or agent behavior for safety, robustness, policy, and production risk before release. We reviewed 18 products from 11 companies on a 100-point evidence score. These 6 earned a place.

Snapshot Oct 19, 202618 products reviewed11 companies

Market noteScores reflect documented task fit, adoption evidence, product detail, pricing and market fit. Vendor claims remain unverified unless independently corroborated. Missing evidence indicates uncertainty, not poor product quality.

  1. 1

    Roark

    Configurable personas and branching phone/WebSocket simulations produce evaluation scores before launch. Failed calls become repeatable tests, with CI/CD quality gates supporting software release workflows. Documented speaker limits and implementation interfaces strengthen product evidence.

    Best for voice AI teams testing conversational edge cases and enforcing release quality gatesFree plan

    Task fit
    40/40
    Adoption
    5/20
    Evidence
    15/15
    Pricing
    11/15
    Market fit
    10/10
  2. 2

    Armature

    Automated tests run complex sessions against MCPs and CLIs to identify broken workflows before users encounter them. Run data, session scoring and root-cause issue grouping support production-risk evaluation for software teams.

    Best for software teams testing MCP and CLI workflows across agent harnesses and models before releaseFree planFrom $50

    Task fit
    40/40
    Adoption
    5/20
    Evidence
    11/15
    Pricing
    11/15
    Market fit
    10/10
  3. 3

    Superagent Red Teaming

    by Superagent

    Attack agents test production AI systems and produce structured findings, evidence, and remediation guidance. Explicit targeting of AI software vendors establishes industry fit. Unlimited free public-repository usage supports the explicit-free-allowance pricing anchor.

    Best for AI software vendors testing enterprise-facing applications for leaks, harmful outputs, and unwanted actionsFree plan

    Task fit
    40/40
    Adoption
    5/20
    Evidence
    11/15
    Pricing
    11/15
    Market fit
    10/10
  4. 4

    Benchmarks

    by Olam Labs

    Explicit pre-deployment behavioral evaluations examine deception, negotiation and long-game consistency, with free public benchmark access.

    Best for AI teams screening multi-agent social behavior before deploymentFree plan

    Task fit
    34/40
    Adoption
    3/20
    Evidence
    11/15
    Pricing
    15/15
    Market fit
    9/10
  5. 5

    AI Red Teaming

    by DeepKeep

    Security teams steer adversarial testing in plain language while an AI red-teaming agent executes and adapts, directly supporting robustness evaluation.

    Best for security teams conducting adaptive model robustness tests

    Task fit
    39/40
    Adoption
    2/20
    Evidence
    12/15
    Pricing
    2/15
    Market fit
    10/10
  6. 6

    Vending-Bench 2

    by Andon Labs Inc.

    A year-long business simulation introduces adversarial suppliers, negotiations and complaints to test sustained agent behavior, alongside explicit safety-benchmarking use cases.

    Best for AI research teams probing long-horizon autonomous-agent robustness

    Task fit
    33/40
    Adoption
    6/20
    Evidence
    10/15
    Pricing
    2/15
    Market fit
    9/10

We reviewed 18 candidate products from 11 companies and published 6 positions. Totals include duplicate company entries and products removed during task, market and evidence review. Each company holds at most one position.