An agent fails on a real customer request. The team changes its prompt or tools, tries the request again and gets a different answer. But the customer’s account has changed, a ticket is now closed or a previous test already issued the refund. A successful second run may say little about whether the change fixed the original problem.
Hue turns recorded agent work into repeatable evaluations. Its public product description connects observability traces with test cases and simulated application environments, so teams can compare versions against the same starting conditions. The point is to evaluate what an agent does across tools and state, as well as what it says. [1]
Built around the gap between a trace and a reliable test
Hue joined YC’s Fall 2026 batch. CEO Aneesh Kethini previously worked on AI products and an AI SRE at Datadog. CTO Josh Le brings engineering experience from companies including Apple and Gusto. Their public YC profile describes a testing platform that uses real requests as the starting point for improving agents. [2]
For a product or engineering leader, that puts the focus on a familiar release question: does the new version handle the cases users actually bring? A trace helps explain a past execution. A repeatable test makes it possible to compare a proposed change with that execution under controlled conditions.
Give each attempt the same starting world
Hue calls the initial state of the simulated applications a world. Its documentation describes constructing that state from production traces and the identities involved in the interaction. Each test attempt gets an isolated instance, so a change made in one attempt does not alter the starting point for another. [3]
The documentation also makes an important boundary explicit: creating a world does not fetch real applications or carry their credentials into the test. It reconstructs state from recorded evidence. If a necessary fact was not captured, the test environment may not contain it. Reproducing an older case therefore depends on retaining sufficient trace context. [3]
This is especially useful for workflows where order matters. Reading an account, changing its status and then posting an update are different actions from merely producing the right final sentence. A stable starting state lets a team examine those differences without repeatedly modifying a live customer record.
Keep the agent’s logic, simulate the applications
Hue’s simulation documentation describes application endpoints that preserve the request and response shapes an agent expects, including errors. The agent keeps its own process, model and prompts; calls are directed to an isolated environment rather than the live service. Calls and resulting state changes are recorded for evaluation. [4]
The guide distinguishes simulated calls from calls that are replayed from a recording. That means coverage matters: a team needs to establish which applications and behaviors are simulated for its workflow, and where a replay provides a narrower test. The presence of a provider logo on a homepage is not enough to establish that every API or edge case is supported. [4]
Test a refund flow beyond the final answer
Hue’s homepage uses a refund interaction to illustrate its product. In that kind of workflow, the useful result includes checking eligibility, selecting the right transaction and making the intended change. A polite confirmation after the wrong action is still a failure. The example is a product demonstration, not a reported customer deployment. [1]
A practical evaluation can compare the original agent with a changed version from the same account and order state. Reviewers can then examine whether the change prevented the unwanted action, introduced a different mistake or simply changed the wording. Cases where no refund should occur belong alongside cases where it should.
The same pattern applies to a support workflow that updates a ticket and informs another system. A team can examine the state left in each application and test interruptions or unexpected responses. The value comes from being able to repeat the task and inspect its effects, rather than assume the agent’s narrative is an accurate record of completion.
Where Hue fits on SOTA2
A ranked comparison is not yet available for Hue.
What we would look for in a pilot
We would begin with a small set of real failures and successful cases from one workflow. Check whether each reconstructed world contains the information the agent originally had. Where the evidence is incomplete, record that limitation rather than treat a passing replay as confirmation of the full live behavior.
Compare versions on task completion, unintended actions and review effort. Keep model variability separate from changes to the test state, and inspect the application records behind the score. A useful evaluation should help the team explain why it is ready to release a change.
The SOTA2 view
Hue is relevant to teams whose agents change state across connected applications. Its strongest idea is turning production evidence into a repeatable environment for checking those changes. The quality of the result depends on the recorded context, simulation coverage and the team’s definition of a successful task.