An agent’s logs can show every tool call and still leave a team unsure what to fix first. One request takes too long, another repeats the same failed lookup, and a third completes the task but costs more than expected. The work of finding a pattern, locating its cause and deciding whether a change is safe often remains manual.
Papaya analyzes production traces to propose improvements to agent quality, latency and cost. Its product connects the evidence behind a recurring problem with a recommendation, then helps the team turn an approved change into something it can review and ship. [1]
A team connecting product operations and engineering
Papaya joined YC’s Fall 2026 batch. CEO Macy Mody previously worked on go-to-market and operations at SafeBase. Co-founders Niyaz Puzhikkunnath and Faiz Vadakkumpadath bring machine-learning and data experience from Amazon. Their public YC description centers on continuously improving agents after deployment. [2]
That focus is useful for teams already serving users. A production agent changes as traffic, tools and customer needs change. Keeping it useful requires a way to decide which problems deserve engineering time and to check whether a proposed fix improved the experience.
Start with the runs behind a recommendation
Papaya’s workflow accepts traces through its SDK, an observability connection or an exported dataset. Connecting the code repository is optional. The product analyzes the runs and ranks findings by expected impact, with the supporting prompt, tools and context available for review. [3]
The documentation’s context-bloat example proposes summarizing earlier conversation turns instead of resending the full history each time. It shows affected runs, an estimated impact and a proposed change. Those numbers illustrate a recommendation; they are not a measured saving for every customer. [3]
For a product owner, the useful feature is the connection between a priority and its evidence. A finding that appears often may affect relatively few users, while a rarer failure may block the main task. Seeing the actual runs helps a team decide which interpretation fits its business.
Find recurring failures and unmet requests
The public product overview describes issue detection, grouping conversations by use case and identifying needs the agent does not meet. It also describes evaluation guidance and alerts when new issues appear. Those views give a team several ways to inspect the same production experience. [1]
A support agent that repeatedly looks up the wrong order status is one illustrative situation. The team needs to understand whether the cause is a tool definition, an input-handling problem or an instruction the model interprets poorly. A pattern across traces makes the investigation more specific than a single customer complaint.
Another useful distinction is between a broken workflow and a request the product never supported. If users repeatedly ask for a capability that is absent, changing a prompt may not solve the problem. Evidence organized by use case can help a product team decide whether to improve the current agent or build something new. These are possible applications of the documented workflow, not named customer outcomes.
Separate reading the code from proposing a change
Papaya’s code-connection documentation describes two GitHub apps. Code Analysis reads selected repositories and their history. Code Changes can create branches and open pull requests for changes the user requests. The company says it does not merge those pull requests; the customer’s reviewers and CI decide what enters the codebase. [4]
That separation gives a team a narrower starting point for an investigation. Repository access can help connect a finding to the version of the code that produced the trace. When the team wants a proposed fix, it can review a normal change with a diff and checks rather than accept an invisible production modification. [4]
Papaya’s workflow then uses new traces to show what changed after a fix and to detect later drift. The operational loop is investigation, review, release and measurement. The recommendation’s initial estimate remains a hypothesis until the team observes the result in its own traffic. [3]
Where Papaya fits on SOTA2
A ranked comparison is not yet available for Papaya.
What we would look for in a pilot
We would bring traces from one workflow and ask the responsible engineers to review the highest-priority findings. Check whether each recommendation identifies a reproducible problem, points to the right evidence and suggests a change the team understands. Include cases where an apparent failure was actually the intended behavior.
For a selected fix, compare task outcomes, response time and cost before and after release. Track review effort and any regressions as well as the headline improvement. Start with the repository permissions needed for that stage of the evaluation.
The SOTA2 view
Papaya is relevant to teams that have observability data but still spend substantial time turning it into improvements. Its useful proposition is a shorter path from a recurring production problem to an explainable, reviewable fix. The result should be judged by the changes a team can validate and maintain.