Building a voice agent is fast. Testing it is not.
A team can prompt an agent in under an hour, then spend hours calling it by hand. Tweak the prompt, call again, and find the regression only when a customer complains. [2][4]
Hamming automates that loop.
It tests voice and chat agents before launch, red-teams what could go wrong and monitors every production call. [1]
When something fails, the failure becomes a regression test, not a story a customer tells. [1]
Thousands of callers before the first real one
Hamming connects to your agent and generates test scenarios automatically. Hundreds of simulated callers, each with a job to do, such as canceling next week’s appointment, call the agent and check whether the job got done. [2][3]
They interrupt, speak with different accents and switch languages. Hamming covers more than 65 languages and accents and can run more than 50,000 test calls at once. [1]
Its red-team suite tries prompt injection, jailbreaks and attempts to pull out personal data. [1]
In production, Hamming scores every call against more than 50 built-in metrics and alerts the team before customers notice. Its automated scores agree with human evaluators 95–96% of the time. [1][3]
When a call goes wrong, one click turns it into a new test, so the next release is checked against it. [1][2]
The first test report takes under ten minutes. Hamming is SOC 2 Type II and offers HIPAA business associate agreements. [1]
Built by people who move metrics with experiments
Sumanyu Sharma grew an AI-powered sales program at Tesla to hundreds of millions of dollars in revenue a year as a senior staff data scientist. Sharma then helped the safety app Citizen grow its monthly users fourfold. [3]
Hamming’s first launches came from Sharma and Marius Buleandra, who ran data infrastructure at Anduril and was a founding engineer at Spell, an MLOps startup acquired by Reddit. [2]
In those jobs they learned that the fastest way to move a metric is to run more experiments. After shipping more than ten AI agents to production in 2023, they saw the same rule hold for reliability: more tests, more reliable agents. [6]
Hamming is part of Y Combinator’s Summer 2024 batch and is based in San Francisco. It has tested more than 4 million calls and monitors more than 10,000 agents for startups, banks and healthtech companies. [1][3]
What this looks like in practice
Bland Labs: 200 test calls before the customer’s first
Bland Labs builds voice agents for companies in healthcare, insurance, legal and home services, and its customers run 1,000 to 10,000 calls a day. [4]
Its engineers could prompt an agent in 30 to 45 minutes, then spend the next two to three hours testing it by hand. Some launches took a week of review. [4]
With Hamming, Bland Labs runs 100 to 200 test calls before a customer starts testing, covers Spanish and other languages without native speakers, and checks healthcare guardrails before deployment. Its deployment cycles are three times faster. [4]
One test caught an agent telling callers it had booked their appointment when it hadn’t. It was fixed before any customer heard it. [4]
Grove AI: every clinical-trial call checked
Grove AI’s agent, Grace, recruits and engages participants for clinical trials. A missed eligibility question can disqualify a participant or put a patient at risk. [5]
Grove’s QA staff could review only 5–10% of calls. With Hamming, Grove checks every call, has analyzed more than 165,000 conversations and gets real-time alerts instead of finding problems days or weeks later. [5]
Before a release, Grove now runs more than 1,000 automated test scenarios, up from about 50 manual tests. [5]
Where Hamming AI ranks
Most teams test a voice agent by calling it themselves. Hamming calls it thousands of times before launch, then checks every real call after it. [1][2]
The bigger picture
Voice agents now book appointments, screen clinical-trial patients and answer insurance questions. [4][5]
Most teams still test them by hand and review a sample of the recordings. [2][5]
Hamming is betting that agents need the same discipline as any other software release: thousands of tests before launch, and every real call checked after it.
The bad call becomes a test, not a customer story.
Written by SOTA2 from Hamming AI’s public materials. Customer experiences and product plans are reported by Hamming AI. Ranking analysis and comparison data are from SOTA2.
Sources
- Hamming AI — testing, red-teaming and production monitoring
- Launch YC — Hamming: automated AI voice agent testing (August 2024)
- Y Combinator — Hamming AI company and founder
- Hamming — Bland Labs case study
- Hamming — Grove AI case study
- Launch YC — Hamming: make your RAG and AI agents reliable (June 2024)
