We tested seven TinyFish products on real company lookups, public browser tasks and controlled page changes. Finding official sources worked better than turning them into an accurate pricing table.
Tested Nov 17–18, 2026API pilotOne account
Results at a glance
Each row has its own unit and sample. Monitor rows distinguish change judgments, uninterrupted checks and post-Resume checks. The post-Resume row covers two instances. Browser results cover the pagination task.
Run usage and billing status
| Product | Observed result | Usage recorded for the tested runs |
|---|---|---|
| TinySearch | Found a usable official product landing and pricing source within five returned links for all 12 named companies. First link passed for 3/12 product and 5/12 pricing queries. | $0 in AgentRouter receipts |
| TinyFetch | Returned 190/192 readable pages and matched 279/343 details for supplied URLs. | $0 in the November 15 study |
| TinyAgent | Returned all four bookstore fields and matched the reference outputs on two dynamic pages. | $0.144 pending for the bookstore; $0.08 and $0.128 shown for the two Playground runs |
| TinyBrowser | Retrieved all four bookstore fields after a retry and all five JavaScript pagination fields on a separate run. | About $0.002 pending across the two bookstore sessions; $0.006 shown for the pagination session |
| TinyMonitor | Correctly judged two edits. In the November 18 audit, matched active controls completed 10/10 and 5/5 checks while REST-resumed instances completed 0/10 and 0/5. | $0.015 pending for an initial baseline and two manual checks |
| TinyResearch | Answered four verifiable questions. Its 734-word report exceeded the 500-word limit. | $2.00 pending for one Deep run |
| TinySet | Returned 12 rows, covering 6/12 requested companies. Passed 26/72 field checks; 2/12 requested rows passed all six. | $1.20 pending for this 12-row run; earlier workflow pilot $0.30 pending |
Search and TinySet include the November 18 real-company study. Other recorded amounts cover the original runs and exclude the November 18 Monitor replication. TinySet Wallet showed 12 rows at $0.10 per row, totaling $1.20 pending. These are usage records, not a total invoice for the review. TinyFish publishes rates per Agent step, Browser minute, Monitor check, Research run and TinySet row. TinyFish pricing
TinyMonitor
| Schedule | Before Pause Resumed instance | Active control Same post-Resume slots | After Resume Resumed instance |
|---|---|---|---|
| Every minute | 3/3 | 10/10 | 0/10 |
| Every two minutes | 2/2 | 5/5 | 0/5 |
Source fetches and webhooks agreed in every cell. Both instances reported active. Manual runs after the observation window read $31. Audit data
| Before Pause | Paused | After Resume | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 10–11 a.m. | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 00 | 01 | 02 |
| Control · 1 min | Control · 1 min, 10:48 AM Pacific. 1 source fetch; 1 webhook in this due minute. | Control · 1 min, 10:49 AM Pacific. 1 source fetch; 1 webhook in this due minute. | Control · 1 min, 10:50 AM Pacific. 1 source fetch; 1 webhook in this due minute. | Control · 1 min, 10:51 AM Pacific. 1 source fetch; 1 webhook in this due minute. | Control · 1 min, 10:52 AM Pacific. 1 source fetch; 1 webhook in this due minute. | Control · 1 min, 10:53 AM Pacific. 1 source fetch; 1 webhook in this due minute. | Control · 1 min, 10:54 AM Pacific. 1 source fetch; 1 webhook in this due minute. | Control · 1 min, 10:55 AM Pacific. 1 source fetch; 1 webhook in this due minute. | Control · 1 min, 10:56 AM Pacific. 1 source fetch; 1 webhook in this due minute. | Control · 1 min, 10:57 AM Pacific. 1 source fetch; 1 webhook in this due minute. | Control · 1 min, 10:58 AM Pacific. 1 source fetch; 1 webhook in this due minute. | Control · 1 min, 10:59 AM Pacific. 1 source fetch; 1 webhook in this due minute. | Control · 1 min, 11:00 AM Pacific. 1 source fetch; 1 webhook in this due minute. | Control · 1 min, 11:01 AM Pacific. 1 source fetch; 1 webhook in this due minute. | Control · 1 min, 11:02 AM Pacific. 1 source fetch; 1 webhook in this due minute. |
| REST Resume · 1 min | REST Resume · 1 min, 10:48 AM Pacific. 1 source fetch; 1 webhook in this due minute. | REST Resume · 1 min, 10:49 AM Pacific. 1 source fetch; 1 webhook in this due minute. | REST Resume · 1 min, 10:50 AM Pacific. 1 source fetch; 1 webhook in this due minute. | REST Resume · 1 min, 10:51 AM Pacific. Paused. | REST Resume · 1 min, 10:52 AM Pacific. Paused. | REST Resume · 1 min, 10:53 AM Pacific. 0 source fetch; 0 webhook in this due minute. | REST Resume · 1 min, 10:54 AM Pacific. 0 source fetch; 0 webhook in this due minute. | REST Resume · 1 min, 10:55 AM Pacific. 0 source fetch; 0 webhook in this due minute. | REST Resume · 1 min, 10:56 AM Pacific. 0 source fetch; 0 webhook in this due minute. | REST Resume · 1 min, 10:57 AM Pacific. 0 source fetch; 0 webhook in this due minute. | REST Resume · 1 min, 10:58 AM Pacific. 0 source fetch; 0 webhook in this due minute. | REST Resume · 1 min, 10:59 AM Pacific. 0 source fetch; 0 webhook in this due minute. | REST Resume · 1 min, 11:00 AM Pacific. 0 source fetch; 0 webhook in this due minute. | REST Resume · 1 min, 11:01 AM Pacific. 0 source fetch; 0 webhook in this due minute. | REST Resume · 1 min, 11:02 AM Pacific. 0 source fetch; 0 webhook in this due minute. |
| Control · 2 min | Control · 2 min, 10:48 AM Pacific. 1 source fetch; 1 webhook in this due minute. | Control · 2 min, 10:49 AM Pacific. No check due. | Control · 2 min, 10:50 AM Pacific. 1 source fetch; 1 webhook in this due minute. | Control · 2 min, 10:51 AM Pacific. No check due. | Control · 2 min, 10:52 AM Pacific. 1 source fetch; 1 webhook in this due minute. | Control · 2 min, 10:53 AM Pacific. No check due. | Control · 2 min, 10:54 AM Pacific. 1 source fetch; 1 webhook in this due minute. | Control · 2 min, 10:55 AM Pacific. No check due. | Control · 2 min, 10:56 AM Pacific. 1 source fetch; 1 webhook in this due minute. | Control · 2 min, 10:57 AM Pacific. No check due. | Control · 2 min, 10:58 AM Pacific. 1 source fetch; 1 webhook in this due minute. | Control · 2 min, 10:59 AM Pacific. No check due. | Control · 2 min, 11:00 AM Pacific. 1 source fetch; 1 webhook in this due minute. | Control · 2 min, 11:01 AM Pacific. No check due. | Control · 2 min, 11:02 AM Pacific. 1 source fetch; 1 webhook in this due minute. |
| REST Resume · 2 min | REST Resume · 2 min, 10:48 AM Pacific. 1 source fetch; 1 webhook in this due minute. | REST Resume · 2 min, 10:49 AM Pacific. No check due. | REST Resume · 2 min, 10:50 AM Pacific. 1 source fetch; 1 webhook in this due minute. | REST Resume · 2 min, 10:51 AM Pacific. No check due. | REST Resume · 2 min, 10:52 AM Pacific. Paused. | REST Resume · 2 min, 10:53 AM Pacific. No check due. | REST Resume · 2 min, 10:54 AM Pacific. 0 source fetch; 0 webhook in this due minute. | REST Resume · 2 min, 10:55 AM Pacific. No check due. | REST Resume · 2 min, 10:56 AM Pacific. 0 source fetch; 0 webhook in this due minute. | REST Resume · 2 min, 10:57 AM Pacific. No check due. | REST Resume · 2 min, 10:58 AM Pacific. 0 source fetch; 0 webhook in this due minute. | REST Resume · 2 min, 10:59 AM Pacific. No check due. | REST Resume · 2 min, 11:00 AM Pacific. 0 source fetch; 0 webhook in this due minute. | REST Resume · 2 min, 11:01 AM Pacific. No check due. | REST Resume · 2 min, 11:02 AM Pacific. 0 source fetch; 0 webhook in this due minute. |
Every minute was observed in full before manual calls. After Resume, the matched active controls completed 10/10 and 5/5 checks. The resumed instances completed 0/10 and 0/5.
In the first controlled run, TinyMonitor correctly ignored an announcement edit and marked the Pro price increase from $20 to $25 relevant. Four scheduled callbacks, including two unchanged checks, arrived 2.9–8.2 seconds into their respective minute slots. This tested change judgment and delivery while the monitor stayed active. Original change events
Source-fetch and webhook counts agreed in each cell. Both Resume calls returned HTTP 200, and a separate GET confirmed both instances active with no reported error. The fixture logged no post-Resume source requests from either instance before the manual calls. Dashboard All checks stayed at six and four baseline-plus-pre-pause entries. Two subsequent manual calls returned HTTP 201 and read the current $31 price, adding one history entry each. Independent source log, webhook receipts and Dashboard checks
These runs support a reproducible gap in scheduled execution after Resume for this account and tested configuration. The 0/12 figure was a scoped interruption test. It cannot serve as an overall Monitor success rate. We have not established the internal cause or measured prevalence across accounts. All four new test instances were paused after the audit. Full audit evidence
Exact method and the earlier comparison
On November 18 we created four fresh Page Monitors under the same account, using ordinary recurring cron expressions * * * * * and */2 * * * *. For each cadence one stayed active and one received status-only REST updates to pause and resume. The observation window ran from 10:48 to 11:03 a.m. Pacific. We paused at 10:50:20, resumed at 10:52:20, and waited until after 11:03 before making manual calls. Every expected minute was observed in full. Plan, request bodies and timestamps · Official lifecycle contract
The November 17 comparison had shown the same gap through Dashboard and REST Resume, at both cadences, using calendar-restricted cron expressions. Its 0/12 figure counted four instances’ first three expected execution times. In the two-minute cases, we made the manual call about 45 seconds into the third due minute. That limited the original observation of that slot. The November 18 repeat used unrestricted recurring schedules and a longer untouched window to address that limitation. Original comparison · Independent recount and replication
| Schedule | Always active Full window | Dashboard Resume First three due times | REST Resume First three due times |
|---|---|---|---|
| Every minute | 11/11 | 0/3 | 0/3 |
| Every two minutes | 6/6 | 0/3 | 0/3 |
One account, six fresh monitors, one fixed observation window. All four resumed instances still read $31 on manual runs.
TinySearch · Find official company pages
Company name + product + official root domain
Requested resultA product landing and a pricing source we can read next
The number is the page's position in the ten links returned by the API. 1 means the first link. Click any path to inspect the accepted source.
| Company and target product | Product landing | Pricing information |
|---|---|---|
| LinearLinear project management | 4 /planlinear.app | 4 /pricinglinear.app |
| NotionNotion workspace | 4 /en-gb/productnotion.com | 1 /pricingnotion.com |
| LoomLoom screen recorder | 1 | 1 /pricingloom.com |
| FigmaFigma Design full seat | 3 /professionalfigma.com | 2 /pricingfigma.com |
| FirecrawlFirecrawl Scrape | 3 /scrapefirecrawl.dev | 3 /pricingfirecrawl.dev |
| ResendResend transactional Email API | 1 /products/transactional-emailsresend.com | 2 /pricingresend.com |
| BrowserbaseBrowserbase cloud browsers | 2 /computer-usebrowserbase.com | 1 /account/billing/plansdocs.browserbase.com |
| ExaExa Lite Search API | 2 /products/searchexa.ai | 1 /docs/admin/pricingexa.ai |
| TiptapTiptap hosted platform with editor | 4 /product/editortiptap.dev | 4 /pricingtiptap.dev |
| RetoolRetool Apps builder seat | 4 /retool.com | 1 /pricingretool.com |
| ElevenLabsElevenCreative Text to Speech | 5 /elevenlabs.io | 2 /pricingelevenlabs.io |
| GleanGlean AI Assistant | 1 /product/assistantglean.com | 5 /glean-core-suite-pricingdocs.glean.com |
We checked company identity, redirects and page content. Relevant official alternatives could pass; a matching domain alone did not. Inputs included the correct root domain. All queries, ten-link lists and decisions ↓
One exact query: reject the other Glean, inspect the right company
Words sent to TinySearch
Glean Glean AI Assistant official pricing plans glean.comMode Web
The ten returned webpage links
- 1Pricing | Glean.ai | Affordable Savingsglean.ai
- 2Glean – Enterprise AI that Works | Agents, Assistant & Searchwww.glean.com
- 3invest with clarity, not surprises.www.glean.com
- 4minutes readwww.glean.com
- 5Glean Core Suitedocs.glean.com
- 6Glean.ai Pricing 2026g2.com
- 7Glean Review: Features, Pricing & Alternatives | HelloStackhellostack.io
- 8Glean Pricing in 2026: Cost, Seats, and Minimumsactivepieces.com
- 9Glean Pricing: How Much Does Glean Cost in 2026?gosearch.ai
- 10What Is Glean AI? Search, Agents, Pricing and Reviews (2026)getmacha.com
Showing links 1–10. Link 1 is the wrong company; link 5 is the accepted official pricing documentation.
The supplied company was glean.com. Link 1 leads to glean.ai. Link 5 describes Glean Core Suite licensing and usage; we did not verify a numeric base-seat starting price in the checked sources.
Dataset selection and scoring, fixed before testing
Loom · FigmaMonthly vs annual and seat types
Browserbase · ExaPlans, usage and request units
ElevenLabs · GleanHosted plans, roles, promotions, sales
We chose four cases per group to exercise different pricing conditions. This is a purposive sample, with no random-sampling or market-wide claim. The cohort, references and query templates froze before the first call.
Each company received two Web queries: company + product + official product page + domain and company + product + official pricing plans + domain. Target paths and expected prices stayed in the evaluator's reference file.
We followed redirects and ignored tracking parameters, fragments and trailing slashes. A product landing had to explain the named product; current pricing documentation could pass the pricing task. A relevant alternative source could pass after independent review. These calls test source finding, with known company domains. They do not score the correctness of a price extracted from those sources.
The task was to find two pages a buyer could use for each company: a landing page explaining the named product, and an official pricing or rate source. We supplied company names, product hints and root domains. TinySearch received search words, without target URL paths or expected prices. It returned ten webpage links for each of 24 Web queries. This tests page finding when the company is already known. Frozen sample, exact queries and scoring rules
All 12 product queries and all 12 pricing queries had a usable official source within the first five returned links. The first link itself passed for 3/12 product queries and 5/12 pricing queries. Position means the link's place in the API's returned list: position 1 is its first link. Each accepted page was checked for company identity, redirects and relevant content. The supplied root domain made this an easier task than discovering an unknown company. All 24 lists and page adjudications
For Glean, the pricing query's first result led to glean.ai, a different company from the supplied glean.com. We rejected it. Glean's own Core Suite pricing documentation appeared at position 5 and described per-user licensing and model-usage charges. We did not verify a numeric starting base-seat price in the inspected references. A same-name company or a page with pricing vocabulary alone did not earn a match. Returned list and ruling · Official Core Suite pricing documentation
A usable official alternative could pass without matching the frozen reference URL character for character. Linear's Plan landing described project management; Figma's Professional landing explained design features and Full seats; Loom's Android screen-recorder landing was valid because we specified no operating system. Retool and ElevenLabs homepages explained the named products. We retained each ruling and the original reference separately. Documentation did not substitute for a product landing; current rate documentation could satisfy the pricing job. Finding these pages did not establish that an extracted price would be right.
The 24 calls completed without retries, with a median provider latency of 1.25 seconds. Each AgentRouter receipt recorded $0. The earlier six-query pilot remains in the original evidence; it is excluded from these 24-query results. New calls and receipts · Earlier pilot
TinyFetch
Firecrawl pricing page
9/9Expected prices matched
Tiptap pricing page
0/6Readable text, six prices missing
In the November 15 study, we supplied 192 exact URLs and checked page text against 343 details. TinyFetch found 46 of 59 prices and 105 of 127 customer quotes. It matched all nine expected prices on Firecrawl’s pricing page and zero of six on Tiptap’s, even though both responses contained readable text. The median run took 1.64 seconds, with a 9.2-second P95. When a workflow uses extracted prices, verify the amounts and billing units against the source. TinyFetch task review · Website benchmark methodology
TinyAgent and TinyBrowser
| Task | TinyAgent Natural-language goal | TinyBrowser Evaluator-written script |
|---|---|---|
| Bookstore product | 4/4 fields | 4/4 fields |
| JavaScript pagination | 5/5 fields | 5/5 fields |
| AJAX year change | 2/2 year rows | Not tested |
The successful bookstore Browser run followed a client retry. Browser script timings exclude provisioning and script authoring.
Both tools retrieved the requested title, price, stock line and URL for It’s Only the Himalayas from a public bookstore fixture. TinyAgent ran for 54 seconds after we removed the unsupported root-level additionalProperties keyword from the output schema. The first TinyBrowser client attempt timed out after 50 seconds. On retry we narrowed the link selector and changed how the client exited on errors. The second attempt returned all four fields in 2.64 seconds, including its CDP connection. The cause of the timeout remains unknown. Bookstore fixture · Run outputs
For JavaScript pagination, both returned the right page URL, card count, first two authors and visible Previous link. TinyAgent followed a natural-language goal. TinyBrowser ran our Playwright script in 2.33 seconds including its CDP connection. That timing excludes provisioning and script authoring. In another TinyAgent run, it switched an AJAX awards table from 2014 to 2015, waited for the new rows and returned the correct winners with their award and nomination counts. We tested public, read-only workflows. Pagination fixture · AJAX fixture · Outputs and reference values
TinyResearch and TinySet
- ✓What total citations count
- ✓What grounding queries represent
- ✓How queries map to cited pages
- ✓How the report relates to rankings
234 words over the requested limit · approximately 47% over
We asked TinyResearch Deep for a brief explaining Bing Webmaster Tools’ AI Performance report. Four answers about citation counts, grounding queries, which pages a query cited and traditional search rankings agreed with Microsoft's documentation. The report finished in 2 minutes 19 seconds and ran to approximately 734 words on a 500-word request. Microsoft's reference · Research run and citation checks
Company coverage
Added outside the requested listAtlassian · Dropbox · Slack · Stripe · Vercel · Zoom
Each check uses the 12 requested companies
Company names were supplied and earned no field point. A correct billing-unit field could pass independently of the amount. Source support required the whole returned plan, amount and billing conditions to agree.
These are the provider's original claims. The reference column is our independent check. Each price links to its source; the missing companies remain visible.
| Company | TinySet returned | Official reference checked | Checks |
|---|---|---|---|
| Linear | Business · $16per user per month, billed monthlyReturned source ↗ | Basic · $12per user / month · monthly billingOfficial reference ↗ | 3/6Wrong amount + entry plan |
| Notion | Plus · $10per month per memberReturned source ↗ | Plus · $12per member / month · monthly billingOfficial reference ↗ | 3/6Annual equivalent; commitment missing |
| Loom | No row returned | Business · $18per user / month · monthly billingOfficial reference ↗ | 0/6Missing requested company |
| Figma | Professional · $16per month per Full seat, billed monthlyReturned source ↗ | Professional · $20per Full seat / month · monthly billingOfficial reference ↗ | 4/6Annual price labeled monthly |
| Firecrawl | No row returned | Hobby · $19per month · monthly billingOfficial reference ↗ | 0/6Missing requested company |
| Resend | No row returned | Pro · $20per month · monthly billingOfficial reference ↗ | 0/6Missing requested company |
| Browserbase | No row returned | Developer · $20per month · monthly billingOfficial reference ↗ | 0/6Missing requested company |
| Exa | Lite Search · $4per 1,000 requests, pay-as-you-go, up to 10 resultsReturned source ↗ | Lite Search · $4per 1,000 requests · pay as you goOfficial reference ↗ | 6/6All six checks passed |
| Tiptap | No row returned | Start · $49per month · hosted platform · monthly billingOfficial reference ↗ | 0/6Missing requested company |
| Retool | Team · $10per month per builder, monthly billingReturned source ↗ | Team · $12per builder / month · monthly billingOfficial reference ↗ | 4/6Annual price labeled monthly |
| ElevenLabs | Starter · $6per month, month-to-monthReturned source ↗ | Starter · $6per month · after promotionOfficial reference ↗ | 6/6All six checks passed |
| Glean | No row returned | No numeric base-seat rate verifiedChecked product and sales referencesOfficial reference ↗ | 0/6Missing requested company |
Monthly toggles were checked before testing. Exa uses a request unit; ElevenLabs excludes the temporary $1 first month. Glean's reference does not establish the absence of a price everywhere. Download the sourced comparison CSV ↓
Exact TinySet prompt, field checks and run limitations
We sent the same company list and product hints directly to TinySet. It did not receive TinySearch's results, reference URL paths or expected prices. The API generated its own ten-column spec. No row was repaired, replaced or retried before scoring.
- Product page: official and explains the named product.
- Pricing page: official rates or a direct sales enquiry for the unpublished case.
- Plan or rate: the requested entry nonfree standard plan.
- Amount and currency: regular monthly USD amount, or Exa's stated request unit.
- Billing basis: period, role or request unit and commitment are explicit.
- Source support: the cited page supports the complete returned price tuple.
Each requested company has six checks. Missing companies receive zero; unrelated replacements receive no credit. Company names earn no field point. Nulls pass only where unavailable in the verified references. The 72 checks are six conditions on 12 companies, not 72 independent samples.
Create exactly one row for each of the 12 real companies listed below, for the named product. Research their current public official websites. Find an official product landing page and its official pricing page, then record a supported regular paid starting price. The purpose is a vendor-comparison table that a buyer can verify from source links. Companies, target products and official root domains: - Linear | Linear project management | linear.app - Notion | Notion workspace | notion.com - Loom | Loom screen recorder | loom.com - Figma | Figma Design full seat | figma.com - Firecrawl | Firecrawl Scrape | firecrawl.dev - Resend | Resend transactional Email API | resend.com - Browserbase | Browserbase cloud browsers | browserbase.com - Exa | Exa Lite Search API | exa.ai - Tiptap | Tiptap hosted platform with editor | tiptap.dev - Retool | Retool Apps builder seat | retool.com - ElevenLabs | ElevenCreative Text to Speech | elevenlabs.io - Glean | Glean AI Assistant | glean.com Return exactly these 10 columns, with these names: 1. company (string, primary key) - use the listed company name. 2. product_url (url) - official landing page explaining the named product. 3. pricing_url (url) - official price/rate page. When the checked official site provides only sales enquiries and no published rate, return the direct sales contact/demo page. 4. plan_or_rate (string) - the lowest nonfree standard paid plan for that product, or the name of its pay-as-you-go rate. Null when no plan is publicly specified. 5. price_amount (number) - numeric USD amount for the regular MONTH-TO-MONTH price, not annual billing. For a pay-as-you-go API quote the amount per stated request unit. Null if no numeric price is published. 6. currency (string) - currency of the published amount. Null with an unpublished amount. 7. billing_basis (string) - include the time period and role/seat or request unit, and any commitment. Use Full seat for Figma Design, builder for Retool Apps, and Lite Search for Exa. For no published amount, state that a quote is needed. 8. pricing_status (string) - published, contact_sales, or unknown. 9. price_source_url (url) - the official page supporting this particular plan/rate, amount and billing basis. May match pricing_url. 10. price_evidence (string) - a short supporting excerpt from that source, at most 18 words. Use regular recurring prices. Exclude zero-price/free plans, free trials, temporary first-month discounts, add-ons and overage charges when choosing the entry plan. For ElevenCreative use its regular price AFTER any first-month promotion. For Resend use transactional Email API plans, not Marketing or add-ons. For Tiptap use the paid hosted platform, not the free open-source Editor. Exa Lite Search is usage-priced, so preserve the request-count unit. Use current official evidence for every filled price. If a value cannot be verified, leave it null or unknown instead of guessing. Use only the 12 listed companies. Do not add other companies or duplicates. The root domains identify the companies, and you must discover the product and pricing paths yourself. Use the dataset name SOTA2 real-company product and price study 2026-11-18 6pk4.
One automatically planned run, one account. It finished about 48 seconds after creation. Wallet showed $1.20 pending for 12 rows at $0.10 each. A seeded-row or alternative-spec recovery was not tested. The evidence excerpt for Exa exceeded the 18-word request; we record that formatting issue separately from the six scored conditions.
For TinySet, the job was a buyer-verifiable vendor table: one row for each of the same 12 companies, with product and pricing URLs, entry paid plan, amount, currency, billing basis and a supporting official source. The prompt required regular month-to-month pricing, excluded free tiers and temporary promotions, specified Figma Full seats and Retool builders, and preserved Exa's request unit. Company names and root domains were supplied; answer paths and prices were withheld. This was an independent TinySet run on the same sample. It did not receive TinySearch's returned links. Exact request and untouched returned rows
The direct API automatically generated a ten-column spec and returned 12 rows. Only six belonged to the requested companies. Loom, Firecrawl, Resend, Browserbase, Tiptap and Glean were missing; Slack, Zoom, Atlassian, Dropbox, Vercel and Stripe appeared instead. We confirmed both the original prompt and the same returned rows in TinyFish's dashboard. There were no duplicate company names and no manual row repairs. Request, API rows and coverage
Six checks applied to every requested row: correct product page, pricing page, plan, amount and currency, billing basis, and source support for the returned price tuple. Missing requested companies earned zero; unrelated replacements earned no credit. The run passed 26/72 checks. Exa and ElevenLabs were the only requested rows passing all six, for 2/12 complete rows. A billing-unit point could pass independently of the amount; the source-support point required the plan, amount and billing conditions to agree together. Per-field decisions and sourced CSV
Figma and Retool illustrate the pricing failure. TinySet labeled $16 per Full seat and $10 per builder as monthly billing. Those were annual-billing equivalents. Before the run, we switched the official pages to monthly and verified $20 and $12 respectively. Notion returned the $10 annual equivalent instead of $12 monthly and omitted the commitment. Linear returned Business at the $16 annual equivalent, although the task asked for the lowest paid plan, Basic at $12 monthly. Links to official pricing pages did not support those claimed tuples. Figma pricing · Retool pricing · Notion pricing · Linear pricing
Exa preserved Lite Search at $4 per 1,000 requests. ElevenLabs selected the regular $6 Starter rate after its $1 first-month offer. The server reported completion about 48 seconds after dataset creation; the next status poll confirmed ready with all rows readable. Wallet recorded $1.20 pending for 12 rows at $0.10 each. This was one automatically planned run in one account. It establishes observed coverage and price errors for this request; the internal cause and performance with seeded rows or another spec remain untested. The earlier three-domain workflow check remains available as historical evidence. Real-company run and scoring · Exa pricing · ElevenLabs pricing · Earlier workflow pilot
What we tested
This API pilot covers seven products on November 17–18, with Fetch results reused from November 15. We used AgentRouter for Search and the earlier Fetch study, TinyFish's direct APIs for the other five products, and its dashboard and Playground for follow-up checks. Search and TinySet received the same purposive 12-company sample on November 18. Four subscription cases were Linear, Notion, Loom and Figma; four API cases were Firecrawl, Resend, Browserbase and Exa; four pricing cases were Tiptap's hosted platform, Retool builder seats, ElevenLabs promotions and Glean sales enquiries. These strata were selected for different price-reading conditions, without a claim that every company fits only one category. Frozen dataset manifest
We fixed the companies, references, query templates and scoring rules at 19:38:45 UTC before the first new Search run at 19:38:59. Reference prices came from official pages; monthly toggles were checked before testing. Additional official pages could qualify after independent content verification, with their rulings saved separately. Prices represent the November 18 observation, exclude taxes and may change. We checked each requested company's identity, not just output row count. The 72 TinySet checks measure six conditions per requested company, not 72 independent samples. Inputs, reference provenance, receipts and all decisions
The 12 companies form a small, hand-selected workflow study. Search was given the correct domains. TinySet had one automatically planned run. These results do not measure open-ended company discovery, repeated-run reliability, or a combined Search-to-TinySet pipeline. The broader review also did not test a combined Search-to-Fetch workflow, topic monitoring, email delivery, authenticated browser workflows, vault or saved browser profiles, enterprise access, or sustained workloads. The browser tasks were public and read-only; the Monitor replication used a synthetic controlled page under one account. Original test evidence · Monitor audit
Test data
Download the exact inputs, returned values and scoring. The files are public and remain available independently of a TinyFish login. Official reference pages can change after the November 18 observation.
- Real-company study ↓
Exact queries, untouched rows, field decisions and billing records.
- Frozen dataset manifest ↓
Twelve selected companies, reference values and predeclared scoring rules.
- Sourced comparison table ↓
Returned values, independent references and every field check.
- Monitor audit ↓
Ordinary recurring schedules, matching controls and source/webhook logs.
- Original API outputs ↓
The original seven-feature pilot and reused November 15 Fetch results.
- Article Markdown ↓
A downloadable copy of the published article and source links.
TinyFish product profile · TinyFish company · TinyFetch Task Review