# Best Tools for Collecting Data from Websites

Source: [Best Tools for Collecting Data from Websites](<https://www.sota2.com/benchmarks/company-website-evidence>)

Collection index: [llms.txt](<https://www.sota2.com/llms.txt>)

Data updated: 2026-11-15

We collect product details, pricing, and customer quotes from company websites for the SOTA2 catalog. We compared tools on that task in two ways: reading a specific page and finding relevant pages starting from a company homepage.

## Which tools we recommend

1. **[AnyAPI](<https://www.sota2.com/products/anyapi-anyapi>)** — You already have the page URL. Read all 192 supplied pages and recovered 340 of 343 expected details. [See measured results](<https://www.sota2.com/benchmarks/company-website-evidence#known-url>).
2. **[Context.dev](<https://www.sota2.com/products/contextdev-context-dev-api>)** — You have only the homepage URL and need results fast. Fastest homepage crawl at 11.75 seconds P95. AnyAPI found the same details for $1.20, compared with $2.874 for Context.dev. [See measured results](<https://www.sota2.com/benchmarks/company-website-evidence#homepage>).
3. **[Exa Search + Contents](<https://www.sota2.com/products/exa-exa-search>)** — You need cited answers to specific questions. Exa found the original source pages and GPT-5.6 Luna extracted the cited answers in a separate test with three questions on two sites. [See measured results](<https://www.sota2.com/benchmarks/company-website-evidence#citation-followup>).

## How we tested

Both tasks looked for the same 343 details, drawn from 82 of the 100 companies: 82 homepage titles, 75 product page titles, 59 prices with billing units, and 127 customer quotes.

1. **Read a specific page.** We supplied 192 specific URLs, such as a pricing or customer story page. The tool had to recover the expected details from the supplied page without following links.
2. **Follow links from the company homepage.** We supplied only the homepage URL for each of 100 companies. Each crawler had to discover the right pages and return the details, using up to 10 pages per site including the homepage.

We checked the text each tool returned against that list. For example, “$500 per month” counts as one pricing detail; “$500” alone doesn’t.

Results arriving after 90 seconds earned no matches. Each tool configuration ran 2 tasks at a time. P95 includes failed attempts and API overhead. Context.dev's P95 excludes the pause between its original and resumed batches.

## Tools tested

- [AnyAPI](<https://www.sota2.com/products/anyapi-anyapi>). Web Scrape and Crawl.
- [Context.dev](<https://www.sota2.com/products/contextdev-context-dev-api>). Page to Markdown and Crawl.
- [Firecrawl](<https://www.sota2.com/products/firecrawl-firecrawl>). Scrape and Crawl.
- [Exa Contents](<https://www.sota2.com/products/exa-exa-search>). Contents, one supplied URL at a time.
- [TinyFish Fetch](<https://www.sota2.com/products/tinyfish-tinyfetch>). Fetch.
- [Search1API](<https://www.sota2.com/products/search1api-search1api>). Crawl and DeepCrawl.

[AgentRouter](<https://www.agentrouter.to/>) charged $13.35695 for page retrieval, including $0.53 for the guided Firecrawl rerun. The results tables show each tool’s charges for the two main tests.

## Read a specific page

| Tool | Pages with text | Company homepage titles | Product page titles | Prices with billing units | Customer quotes | Total expected details found | P95 seconds | API charge (USD) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| AnyAPI | 192/192 | 82/82 | 75/75 | 56/59 | 127/127 | 340/343 | 7.19 | 0.13495 |
| Context.dev | 191/192 | 79/82 | 72/75 | 54/59 | 122/127 | 327/343 | 3.07 | 0.576 |
| Firecrawl | 190/192 | 73/82 | 67/75 | 49/59 | 119/127 | 308/343 | 5.00 | 0.960 |
| Exa | 190/192 | 73/82 | 69/75 | 43/59 | 114/127 | 299/343 | 0.94 | 0.190 |
| TinyFish | 190/192 | 64/82 | 64/75 | 46/59 | 105/127 | 279/343 | 9.20 | 0 |
| Search1API | 192/192 | 37/82 | 40/75 | 37/59 | 86/127 | 200/343 | 4.01 | 0.192 |

## Follow links from the company homepage

| Tool | Sites with at least one readable page | Company homepage titles | Product page titles | Prices with billing units | Customer quotes | Total expected details found | P95 seconds | API charge (USD) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Context.dev | 92/100 | 79/82 | 43/75 | 40/59 | 72/127 | 234/343 | 11.75 | 2.874 |
| AnyAPI | 92/100 | 79/82 | 43/75 | 40/59 | 72/127 | 234/343 | 27.30 | 1.20000 |
| Firecrawl | 89/100 | 70/82 | 26/75 | 3/59 | 58/127 | 157/343 | 31.56 | 4.700 |
| Search1API | 89/100 | 27/82 | 14/75 | 13/59 | 32/127 | 86/343 | 23.90 | 2.00 |

AnyAPI and Context.dev matched the same details and source URLs across all 100 sites. The AgentRouter receipts did not identify AnyAPI’s upstream provider. [AnyAPI routing documentation](<https://getanyapi.com/docs>).

## Why the starting URL matters: Firecrawl on Momentic

We looked for seven prices with their billing units on Momentic’s pricing page, including $500 per month and $0.005 per minute. Given the pricing URL, Firecrawl found all seven in the returned text. Given only the homepage, it returned the homepage and nine documentation pages, missed the pricing page, and found none of the seven prices.

## Can tools return facts with sources?

The main test checks what the returned pages contain. These three questions on two sites check whether a workflow can answer a question and cite the page that supports each fact.

- **Only a homepage:** Search the official domain for the question. Exa Search + Contents found the original source for all three questions and the shared extractor returned 16 of 16 expected facts with supporting citations.
- **Already have the source page:** Start by extracting from the known page and verify the cited URL. The supplied-page control returned 16 of 16 expected facts with supporting citations. In the original pilot, Context.dev passed 15 and Firecrawl Scrape passed 5.

### Finding answers and their sources from a homepage

For the Roo question, Exa returned the original customer story first. Firecrawl Search found three correct, citable facts on other Momentic pages. Context.dev’s homepage crawl scored 0/4 on that question.

| Workflow | Salient products | Momentic pricing | Roo customer story | Total facts with supporting citations | Cost for 3 questions (USD) |
| --- | --- | --- | --- | --- | --- |
| Exa search + contents | 5/5 | 7/7 | 4/4 | 16/16 | ≈ $0.06584 (AgentRouter receipts $0.051 + model est. $0.01484) |
| Firecrawl Search | 5/5 | 7/7 | 3/4 | 15/16 | ≈ $0.20215 (AgentRouter receipts $0.18 + model est. $0.02215) |
| Context.dev crawl | 5/5 | 7/7 | 0/4 | 12/16 | ≈ $0.08471 (AgentRouter receipts $0.06 + model est. $0.02471) |
| Supplied page control | 5/5 | 7/7 | 4/4 | 16/16 | ≈ $0.0127 (Recorded page fetches $0.009 + model est. $0.0037) |

We gave each homepage workflow the same three questions. Search was restricted to the company’s domain. Each workflow passed up to 10 pages to gpt-5.6-luna for extraction. The supplied-page control reused recorded page text.

A fact passed when the answer matched the expected detail and the cited official page supported it. A separate official page could support the same claim.

The follow-up used the same three questions on two sites. The original Firecrawl Agent run used its own extraction and timing, so its result is listed alongside the original pilot.

Each total covers all three questions. Retrieval charges come from AgentRouter receipts, including recorded fetches for the supplied-page control. We counted each reused crawl once. Model estimates use recorded token counts and OpenAI list rates, including cache writes. The totals exclude the infrastructure that ran the extractor. [OpenAI’s GPT-5.6 Luna rates](<https://developers.openai.com/api/docs/models/gpt-5.6-luna>).

[AgentRouter](<https://www.agentrouter.to/>) charged $0.291 for the new retrieval calls. [Answers, citations, and costs (JSON)](<https://www.sota2.com/benchmarks/company-website-citation-followup-2026-11-15.json>).

### Original citation pilot

The original pilot used Firecrawl Agent to check whether an agent could find and cite facts starting at a homepage. It returned 16 of 16 expected facts with supporting citations in about 46–96 seconds per task. Firecrawl Scrape and Context.dev received the source URLs.

The citation rules differ: with a supplied page, the citation had to point to that exact page. Starting from a homepage, any official company page that supported the fact could pass.

| Tool | Starting input | Expected facts found | Facts with passing citations |
| --- | --- | --- | --- |
| Firecrawl Scrape | Given the source page | 14/16 | 5/16 |
| Context.dev | Given the source page | 15/16 | 15/16 |
| Firecrawl Agent | Given only the homepage | 16/16 | 16/16 |

For Salient, Firecrawl Scrape cited product subpages instead of the supplied product index, so those five citations did not pass. Firecrawl Agent’s product subpage citations passed under the homepage rule.

For the Roo story, Firecrawl Scrape found all four expected facts but cited Roo’s homepage for all of them. The required source was [Momentic’s customer story](<https://momentic.ai/customers/roo>). [See all three pilot results](<https://www.sota2.com/benchmarks/company-website-evidence#structured-pilot>).

## Conclusion

### You have the exact page URL

Read the text on a specific page, such as a pricing page or customer story. We tested 192 URLs.

- **API cost.** TinyFish Fetch. $0.00 charged in this run, with 279/343 details found. AnyAPI was the cheapest paid option at $0.13495.
- **Coverage.** AnyAPI. 340/343 details found. It returned readable text for all 192 URLs.
- **Speed.** Exa Contents. 0.94 seconds at P95.

**If you need citations.** Read the source page first. With the shared model, the supplied-page control returned 16/16 facts with supporting citations. [See citation results for known pages](<https://www.sota2.com/benchmarks/company-website-evidence#citation-known-page>).

[See single-page results](<https://www.sota2.com/benchmarks/company-website-evidence#known-url>)

### You only have the company homepage

Find relevant pages across the site. We tested 100 companies with a limit of 10 pages per site.

- **API cost.** AnyAPI. $1.20 for all 100 sites. Context.dev cost $2.874 for the same coverage.
- **Coverage.** AnyAPI and Context.dev tied. Both found 234/343 expected details on the same source pages.
- **Speed.** Context.dev. 11.75 seconds at P95.

**If you need citations.** Exa Search + Contents with GPT-5.6 Luna. 16/16 facts with supporting citations across three questions on two sites, at an estimated $0.066 total. [See citation results from a homepage](<https://www.sota2.com/benchmarks/company-website-evidence#citation-followup>).

[See homepage crawl results](<https://www.sota2.com/benchmarks/company-website-evidence#homepage>)

## Run evidence and citation

Cite the HTML sections above for the applicable input and result. Example company profiles: [Salient](<https://www.sota2.com/companies/salient>), [Momentic](<https://www.sota2.com/companies/momentic>).

- [Summary scorecard (JSON)](<https://www.sota2.com/benchmarks/company-website-evidence-2026-11-15.json>). Coverage, speed, and API charges by tool, plus the leaders for each metric.
- [Individual runs (JSON)](<https://www.sota2.com/benchmarks/company-website-retrieval-2026-11-15.json>). The URL each tool received, the pages it returned, the details it found, and the time and cost per run.
- [Answers, citations, and costs (JSON)](<https://www.sota2.com/benchmarks/company-website-citation-followup-2026-11-15.json>). The answers, source links, scores, and costs for the three-question test.
- [Original citation test (JSON)](<https://www.sota2.com/benchmarks/company-website-structured-pilot-2026-11-15.json>). The prompts, returned facts, and source scores from the original Firecrawl and Context.dev test.

## Where tested tools appear in SOTA2 rankings

Use this benchmark as evidence for the capabilities listed beside each ranking.

- [Discover Agent Services](<https://www.sota2.com/rankings/industries/agent-native/discover-agent-services>) · AnyAPI’s Web Scrape and Crawl APIs
- [Access Data & Knowledge](<https://www.sota2.com/rankings/industries/agent-native/access-data-and-knowledge>) · Firecrawl’s Scrape, Crawl, Search, and Agent
- [Browse & Act on the Web](<https://www.sota2.com/rankings/industries/agent-native/browse-and-act-on-web>) · Firecrawl’s Scrape, Crawl, Search, and Agent
- [Answer Domain Questions](<https://www.sota2.com/rankings/industries/agent-native/answer-domain-questions>) · Exa Contents for reading pages and Exa Search with Contents for finding answers with citations and Context.dev’s page retrieval, website crawls, and JSON extraction
