SOTA2 first-party benchmark
Best Tools for Collecting Data from Websites
We collect product details, pricing, and customer quotes from company websites for the SOTA2 catalog. We compared tools on that task in two ways: reading a specific page and finding relevant pages starting from a company homepage.
Which tools we recommend
AnyAPI
You already have the page URLRead all 192 supplied pages and recovered 340 of 343 expected details.
Context.dev
You have only the homepage URL and need results fastFastest homepage crawl at 11.75 seconds P95. AnyAPI found the same details for $1.20, compared with $2.874 for Context.dev.
Exa Search + Contents
You need cited answers to specific questionsExa found the original source pages and GPT-5.6 Luna extracted the cited answers in a separate test with three questions on two sites.
How we tested
Both tasks looked for the same 343 details, drawn from 82 of the 100 companies: 82 homepage titles, 75 product page titles, 59 prices with billing units, and 127 customer quotes.
- 1. Read a specific page
- We supplied 192 specific URLs, such as a pricing or customer story page. The tool had to recover the expected details from the supplied page without following links.
- 2. Follow links from the company homepage
- We supplied only the homepage URL for each of 100 companies. Each crawler had to discover the right pages and return the details, using up to 10 pages per site including the homepage.
We checked the text each tool returned against that list. For example, “$500 per month” counts as one pricing detail; “$500” alone doesn’t.
Tools tested AnyAPI · Context.dev · Firecrawl · Exa Contents (API docs) · TinyFish Fetch · Search1API
Scoring rules and timing
We selected the sample from our catalog work. Starting from a homepage, a price could come from another pricing page on the same host, and a quote could appear anywhere on that host. We counted repeated spellings of the same price and unit on a page once.
We ran 2 tasks at a time per tool configuration through AgentRouter for external tools. Results arriving after 90 seconds earned no matches. P95 is the time by which 95% of measured tasks ended, including failed attempts and API overhead.
Exa Contents and TinyFish Fetch took part in the main single-page test. The separate citation test also used Exa Search.
Summary scorecard (JSON). Coverage, speed, and API charges by tool, plus the leaders for each metric.
Individual runs (JSON). The URL each tool received, the pages it returned, the details it found, and the time and cost per run.
AgentRouter charged $13.35695 for page retrieval, including $0.53 for the guided Firecrawl rerun. The results tables show each tool’s charges for the two main tests.
Read a specific page 192 pages
Goal: recover homepage titles, product page titles, prices with billing units, and customer quotes from the pages we supplied. AnyAPI found 340 of 343 expected details and returned text for all 192 pages.
Scroll the table to see all columns.
| Tool or route | Pages with text | Company homepage titles | Product page titles | Prices with billing units | Customer quotes | Total expected details found | P95 task time | API charge |
|---|---|---|---|---|---|---|---|---|
| AnyAPI | 192/192 | 82/82 | 75/75 | 56/59 | 127/127 | 340/343 | 7.19 s | $0.13495 |
| Context.dev | 191/192 | 79/82 | 72/75 | 54/59 | 122/127 | 327/343 | 3.07 s | $0.576 |
| Firecrawl | 190/192 | 73/82 | 67/75 | 49/59 | 119/127 | 308/343 | 5.00 s | $0.96 |
| Exa | 190/192 | 73/82 | 69/75 | 43/59 | 114/127 | 299/343 | 0.94 s | $0.19 |
| TinyFish | 190/192 | 64/82 | 64/75 | 46/59 | 105/127 | 279/343 | 9.20 s | $0.00 |
| Search1API | 192/192 | 37/82 | 40/75 | 37/59 | 86/127 | 200/343 | 4.01 s | $0.192 |
A tool can return text but miss some details. A single pricing page can contain several prices, each counted separately.
Follow links from the company homepage 100 companies
Goal: start with only the homepage URL, then find homepage and product page titles, prices with billing units, and customer quotes within 10 pages. AnyAPI and Context.dev tied for the most expected details found (234/343).
Product page titles
Prices with billing units
Customer quotes
Scroll the table to see all columns.
| Tool or route | Sites with at least one readable page | Company homepage titles | Product page titles | Prices with billing units | Customer quotes | Total expected details found | P95 task time | API charge |
|---|---|---|---|---|---|---|---|---|
| Context.dev | 92/100 | 79/82 | 43/75 | 40/59 | 72/127 | 234/343 | 11.75 s | $2.874 |
| AnyAPI | 92/100 | 79/82 | 43/75 | 40/59 | 72/127 | 234/343 | 27.30 s | $1.20 |
| Firecrawl | 89/100 | 70/82 | 26/75 | 3/59 | 58/127 | 157/343 | 31.56 s | $4.70 |
| Search1API | 89/100 | 27/82 | 14/75 | 13/59 | 32/127 | 86/343 | 23.90 s | $2.00 |
AnyAPI and Context.dev matched the same details and source URLs across all 100 sites. The AgentRouter receipts did not identify AnyAPI’s upstream provider. AnyAPI routing documentation.
Returning one readable page counted toward the sites column, even if that page contained none of the expected details.
We looked for seven prices with their billing units on Momentic’s pricing page, including $500 per month and $0.005 per minute. Given the pricing URL, Firecrawl found all seven in the returned text. Given only the homepage, it returned the homepage and nine documentation pages, missed the pricing page, and found none of the seven prices.
Given the pricing page
https://momentic.ai/pricing7 of 7 expected prices found with their units
1 page · 1.91 s · $0.005
See the URLs returned
Given only the homepage
https://momentic.ai/0 of 7 expected prices found with their units
10 pages · 20.84 s · $0.05
See the URLs returned
- /
- /docs/ai/agents
- /docs/cli-reference/momentic/commands/init
- /docs/cli-reference/momentic/commands/install-browsers.md
- /docs/coding-agents/skills
- /docs/platforms/android/emulators.md
- /docs/reference/commands/visual-diff.md
- /docs/reference/mobile-commands/add-file
- /docs/reference/mobile-commands/clear-camera
- /docs/reference/mobile-commands/module
Crawl setup and reruns
In a separate 12-company test, we ran the same Firecrawl crawl with instructions to prioritize products, prices, and customer evidence. Pricing hits went from 0 to 15 out of 22. Homepage-title hits went from 12 to 0 out of 12. The instructions changed which evidence surfaced under the same budget.
We resumed 191 Context.dev single-page calls and 93 homepage crawls after the initial provider quota interruption. Inputs and limits stayed the same. The scorecard combines the original and resumed tasks. P95 measures individual task times and excludes the pause between batches.
Can tools return facts with sources?
The main test checks what the returned pages contain. These three questions on two sites check whether a workflow can answer a question and cite the page that supports each fact.
- Only a homepage
- Search the official domain for the question. Exa Search + Contents found the original source for all three questions and the shared extractor returned 16 of 16 expected facts with supporting citations.
- Already have the source page
- Start by extracting from the known page and verify the cited URL. The supplied-page control returned 16 of 16 expected facts with supporting citations. In the original pilot, Context.dev passed 15 and Firecrawl Scrape passed 5.
Finding answers and their sources from a homepage
For the Roo question, Exa returned the original customer story first. Firecrawl Search found three correct, citable facts on other Momentic pages. Context.dev’s homepage crawl scored 0/4 on that question.
Scroll the table to see all columns.
| Workflow | Salient products | Momentic pricing | Roo customer story | Total | Cost for 3 questions (USD) |
|---|---|---|---|---|---|
| Exa search + contents | 5/5 | 7/7 | 4/4 | 16/16 | ≈ $0.06584AgentRouter receipts $0.051Model est. $0.01484 |
| Firecrawl Search | 5/5 | 7/7 | 3/4 | 15/16 | ≈ $0.20215AgentRouter receipts $0.18Model est. $0.02215 |
| Context.dev crawl | 5/5 | 7/7 | 0/4 | 12/16 | ≈ $0.08471AgentRouter receipts $0.06Model est. $0.02471 |
| Supplied page control | 5/5 | 7/7 | 4/4 | 16/16 | ≈ $0.0127Recorded page fetches $0.009Model est. $0.0037 |
The follow-up used the same three questions on two sites. The original Firecrawl Agent run used its own extraction and timing, so its result is listed alongside the original pilot.
How we compared search and crawl
We gave each homepage workflow the same three questions. Search was restricted to the company’s domain. Each workflow passed up to 10 pages to gpt-5.6-luna for extraction. The supplied-page control reused recorded page text.
A fact passed when the answer matched the expected detail and the cited official page supported it. A separate official page could support the same claim.
Each total covers all three questions. Retrieval charges come from AgentRouter receipts, including recorded fetches for the supplied-page control. We counted each reused crawl once. Model estimates use recorded token counts and OpenAI list rates, including cache writes. The totals exclude the infrastructure that ran the extractor.
AgentRouter charged $0.291 for the new retrieval calls. See OpenAI’s GPT-5.6 Luna rates and cache pricing.
Original pilot with Firecrawl Agent, Firecrawl Scrape, and Context.dev
The original pilot used Firecrawl Agent to check whether an agent could find and cite facts starting at a homepage. It returned 16 of 16 expected facts with supporting citations in about 46–96 seconds per task. Firecrawl Scrape and Context.dev received the source URLs.
The citation rules differ: with a supplied page, the citation had to point to that exact page. Starting from a homepage, any official company page that supported the fact could pass.
| Task | Firecrawl ScrapeGiven the source page | Context.devGiven the source page | Firecrawl AgentGiven only the homepage |
|---|---|---|---|
| List Salient’s products and their uses | 5/5 expected facts found0/5 with passing citations | 5/5 expected facts found5/5 with passing citations | 5/5 expected facts found5/5 with passing citations |
| Find Momentic’s prices and billing rules | 5/7 expected facts found5/7 with passing citations | 6/7 expected facts found6/7 with passing citations | 7/7 expected facts found7/7 with passing citations |
| Find the results Roo reported | 4/4 expected facts found0/4 with passing citations | 4/4 expected facts found4/4 with passing citations | 4/4 expected facts found4/4 with passing citations |
| Total across all three tasks | 14/16 expected facts found5/16 with passing citations | 15/16 expected facts found15/16 with passing citations | 16/16 expected facts found16/16 with passing citations |
How the citation rule affected the Salient result
For Salient, Firecrawl Scrape cited product subpages instead of the supplied product index, so those five citations did not pass. Firecrawl Agent’s product subpage citations passed under the homepage rule.
See the facts we checked in the pilot
List Salient’s products and their uses
- Taylor handles collections
- Marshall automates audits
- Flyn handles insurance claims for totaled vehicles
- Alex handles chargebacks and disputes
- Melanie automates chargeoff decisions
Find Momentic’s prices and billing rules
- Scale costs $500 per month and bills in advance
- Runs pause when the balance reaches zero
- Unused included usage does not roll over
- Scale top-ups range from $50 to $1,000
- Added funds do not expire
- Hosted browser has a $0.005 per-minute meter
- iOS simulator has a $0.1875 per-minute meter
Find the results Roo reported
- Speaker's full name and role
- Expected savings are 200+ hours per quarter
- Test authoring is 4x faster
- Flaky-test stabilization and maintenance time down about 50%
Firecrawl Scrape returned all four facts about Roo but cited Roo’s homepage for each. The link needed to point to Momentic’s customer story.
- AgentRouter receipts totaled $0.12 for the 6 single-page calls. The 3 Firecrawl Agent calls showed $0.00 charged in this run.
Conclusion
You have the exact page URL
Read the text on a specific page, such as a pricing page or customer story. We tested 192 URLs.
- API cost
- TinyFish Fetch$0.00 charged in this run, with 279/343 details found. AnyAPI was the cheapest paid option at $0.13495.
- Coverage
- AnyAPI340/343 details found. It returned readable text for all 192 URLs.
- Speed
- Exa Contents0.94 seconds at P95.
If you need citations Read the source page first. With the shared model, the supplied-page control returned 16/16 facts with supporting citations. See citation results for known pages.
See single-page resultsYou only have the company homepage
Find relevant pages across the site. We tested 100 companies with a limit of 10 pages per site.
- API cost
- AnyAPI$1.20 for all 100 sites. Context.dev cost $2.874 for the same coverage.
- Coverage
- AnyAPI and Context.dev tiedBoth found 234/343 expected details on the same source pages.
- Speed
- Context.dev11.75 seconds at P95.
If you need citations Exa Search + Contents with GPT-5.6 Luna. 16/16 facts with supporting citations across three questions on two sites, at an estimated $0.066 total. See citation results from a homepage.
See homepage crawl resultsHow to cite this benchmark
In SOTA2’s November 15, 2026 company website benchmark, AnyAPI and Context.dev tied for the most expected details (234/343) starting from 100 company homepages with a 10-page limit per site. In a separate 192-URL test, AnyAPI found the most expected details when given the exact pages (340/343 checks).
Best Tools for Collecting Data from Websites · Summary scorecard (JSON)