# Context.dev API Feature Review

Source: [Context.dev API Feature Review](<https://www.sota2.com/benchmarks/context-dev-api>)

Collection index: [llms.txt](<https://www.sota2.com/benchmarks/llms.txt>)

Data updated: 2026-11-16

We tested what Context.dev could extract from known webpage URLs. We also tested its dedicated G2 review parser, PDF table extraction with and without OCR, and scheduled page monitoring.

Tested November 16, 2026. 27 initial requests and 16 reruns.

## Results at a glance

| Test | Matched |
| --- | --- |
| Software pricing fields | 13/13 |
| App Store listings and absences | 8/8 |
| G2 review IDs and ratings | 13/13 |
| Store prices and stock | 5/8 |
| Text PDF cells | 9/12 |
| Scanned Census PDF cells (OCR) | 1/4 |
| Changes held through the next poll | 2/2 |

## Public webpages · First attempt

We scored fields separately and counted a page as correct only if every answer matched the source. For App Store pages the check was whether the app was listed in the tested country.

| Task | Pages with every answer correct | Field checks passed |
| --- | --- | --- |
| Software pricing | 4/4 | 13/13 |
| Hotel room details | 3/4 | 7/11 |
| Store prices and stock | 1/3 | 5/8 |
| App Store visibility | 8/8 | 8/8 |

### Software pricing

- [Momentic](<https://momentic.ai/pricing>). Expected: Standard step $0.0125. Browser per minute $0.005 Standard, $0.0067 Large. Observed: All 3 values matched.
- [Linear](<https://linear.app/pricing>). Expected: Basic $10 and Business $16 per user per month. Business billed annually. Enterprise custom pricing, so no number. Observed: All 4 values matched, including an explicit null for Enterprise.
- [Resend](<https://resend.com/pricing>). Expected: Pro $20/month, 50,000 included emails/month and $0.90 per 1,000 extra emails. Observed: All 3 values matched.
- [PostHog](<https://posthog.com/pricing>). Expected: 1 Free project, 6 pay-as-you-go projects and 2 years of pay-as-you-go retention. Observed: All 3 values matched.

### Hotel room details

- [Park Hyatt Kuala Lumpur](<https://www.hyatt.com/park-hyatt/en-US/kulph-park-hyatt-kuala-lumpur/rooms>). Expected: Minimum areas of 54 m² for Tower King and 104 m² for Park Suite, plus the bedroom photo URL for each. Observed: Page access error. No requested values returned (0/4).
- [The Singapore EDITION](<https://www.marriott.com/en-us/hotels/sineb-the-singapore-edition/photos/>). Expected: Garden Premier King photo and Premier Suite living room photo, matched by gallery caption and image asset. Observed: Both image assets matched (2/2).
- [The Hoxton, Shoreditch](<https://thehoxton.com/london/shoreditch/bedrooms/>). Expected: Shoebox 12 m², Cosy Up 20 m², and Cosy Up maximum occupancy of 2 adults. Observed: All 3 values matched.
- [The Ludlow](<https://ludlowhotel.com/sleep/2/-lofts/>). Expected: Loft King 325 ft² and Ludlow Loft 405 ft². Observed: Both areas matched (2/2).

### Store prices and stock

- [Allbirds Mist](<https://www.allbirds.com/products/mens-tree-runners-mist>). Expected: Sizes 8 and 9 both $100. Size 8 out of stock, size 9 in stock. Observed: Both prices and size 9 stock matched. Size 8 stock returned null (3/4).
- [Ugmonk Walnut base kit](<https://ugmonk.com/products/analog-starter-kit>). Expected: $69 and in stock. Observed: Both values matched (2/2).
- [Ugmonk Walnut kit with refills](<https://ugmonk.com/products/analog-daily-focus-kit-3-refills-walnut>). Expected: $89 and in stock. Observed: Page access error. No requested values returned (0/2).

### App Store visibility

- [Spotify · US](<https://apps.apple.com/us/app/spotify-music-and-podcasts/id324684580>). Expected: Public listing visible. Observed: Listing returned for the US storefront.
- [Spotify · UK](<https://apps.apple.com/gb/app/spotify-music-and-podcasts/id324684580>). Expected: Public listing visible. Observed: Listing returned for the UK storefront.
- [Venmo · US](<https://apps.apple.com/us/app/venmo/id351727428>). Expected: Public listing visible. Observed: Listing returned for the US storefront.
- [Venmo · UK](<https://apps.apple.com/gb/app/venmo/id351727428>). Expected: No public listing. Observed: Source returned HTTP 404.
- [BBC iPlayer · UK](<https://apps.apple.com/gb/app/bbc-iplayer/id416580485>). Expected: Public listing visible. Observed: Listing returned for the UK storefront.
- [BBC iPlayer · US](<https://apps.apple.com/us/app/bbc-iplayer/id416580485>). Expected: No public listing. Observed: Source returned HTTP 404.
- [PayPay · Japan](<https://apps.apple.com/jp/app/paypay/id1435783608>). Expected: Public listing visible. Observed: Listing returned for the Japan storefront.
- [PayPay · US](<https://apps.apple.com/us/app/paypay/id1435783608>). Expected: No public listing. Observed: Source returned HTTP 404.

## Same request, two reruns

We reran five samples with misses and three passing controls through AgentRouter. Three of the five missed samples returned every field correctly in both reruns. Both reruns scored the same on all five samples.

| Test sample | First run | Rerun 1 | Rerun 2 | Observed |
| --- | --- | --- | --- | --- |
| Park Hyatt Kuala Lumpur | 0/4 | 2/4 | 2/4 | Both room sizes matched. The API returned no photo links. |
| Ugmonk kit with refills | 0/2 | 2/2 | 2/2 | Page access error first. Both reruns found $89 and in stock. |
| Allbirds Mist | 3/4 | 4/4 | 4/4 | Size 8 stock was missing initially. Both reruns found it out of stock. |
| Federal Reserve H.4.1 | 3/4 | 3/4 | 3/4 | No Wednesday Reserve Bank credit value on any of the three attempts. |
| BEA GDP | 2/4 | 4/4 | 4/4 | Both reruns found the missing Q2 GDP (3.8%) and investment (−13.8%) figures. |

Both Hyatt reruns returned the requested 54 m² and 104 m² areas and returned null for both photo links. All three controls (Singapore EDITION, Ugmonk base kit, BLS CPI) passed on all attempts.

Independent checks found the same merchant reference values and identical Fed and BEA PDF files. These results describe the full route and do not identify which part of it caused the changes. To investigate, repeat matched direct and gateway calls on a fixed source and compare each gateway call's raw upstream response with its tool output. Gateway queue rejections were logged separately from scored calls.

## G2 reviews · IDs, ratings and dates

We sent each product review-list URL to the Context.dev G2 parser and joined its output to public G2 reviews by ID. Both products had fewer than 25 public reviews. All 13 IDs and ratings matched and 10 of 12 comparable publication dates matched in Pacific time. One Umami review displayed only an edit date and counted for ID and rating only.

| G2 product | ID and rating matched | Date matched (Pacific) |
| --- | --- | --- |
| Plausible Analytics | 7/7 | 6/7 |
| Umami | 6/6 | 4/5 |

### Plausible Analytics

| Review ID | Stars on G2 and in the API | Date on G2 | API date (Pacific) |
| --- | --- | --- | --- |
| 14059296 | 5 | 2026-11-11 | 2026-11-11 |
| 13871456 | 5 | 2026-10-27 | 2026-10-27 |
| 13850818 | 5 | 2026-10-23 | 2026-10-23 |
| 13628585 | 5 | 2026-09-28 | 2026-09-28 |
| 9765355 | 4.5 | 2024-06-22 | 2024-06-08 |
| 8435208 | 5 | 2023-08-09 | 2023-08-09 |
| 6833777 | 5 | 2022-07-10 | 2022-07-10 |

### Umami

| Review ID | Stars on G2 and in the API | Date on G2 | API date (Pacific) |
| --- | --- | --- | --- |
| 12371028 | 4.5 | 2026-02-17 | 2026-02-17 |
| 10032446 | 4.5 | 2024-08-21 | 2024-08-21 |
| 9928232 | 4.5 | 2024-07-26 | 2024-07-26 |
| 9926970 | 5 | 2024-07-26 | 2024-07-26 |
| 7184555 | 3.5 | Updated 2023-03-28 | 2022-10-10 |
| 7597495 | 5 | 2023-01-19 | 2023-01-15 |

### Earlier test of four named reviews

Both the generic JSON route and the dedicated G2 parser got 0/4 complete answers. The dedicated parser returns up to 25 recent reviews per product.

- [Slack](<https://www.g2.com/products/slack/reviews>). Expected: “Intuitive Team Collaboration, Though Search and Alerts Need Work.” 4.5 stars, published 2026-11-04, updated 2026-11-08. Observed: Generic scraper: page access error. G2 parser: target absent from 25 results.
- [monday CRM](<https://www.g2.com/products/monday-crm/reviews>). Expected: “monday CRM with Unleashed made our sales process smooth.” 4.5 stars, published 2026-11-09. Observed: Generic scraper: page access error. G2 parser: target absent from 25 results.
- [Linear](<https://www.g2.com/products/linear/reviews>). Expected: “Linear hits the right balance of performance and features.” 5 stars, published 2026-08-10. Observed: Generic scraper: page access error. G2 parser: target absent from 25 results.
- [Plausible Analytics](<https://www.g2.com/products/plausible-analytics/reviews>). Expected: “easy to use and secure tool for site tracking.” 4.5 stars, published 2024-06-22. Observed: Generic scraper: both fields null. G2 parser: found the target among 7 results, returned 4.5 stars and 2024-06-08.

## PDF tables and OCR

We asked for four specific table cells per PDF and checked the value, row, column, period and unit. The three text PDFs matched 9/12 cells on the first run. The Census scan returned no cells by URL and got 1/4 when we uploaded page 24 with OCR enabled.

- [BLS CPI · page 2, Table A](<https://www.bls.gov/news.release/archives/cpi_01132026.pdf#page=2>). Expected: October all-items monthly change unpublished. December monthly change 0.3%, annual change 2.7%, gasoline annual change −3.4%. Observed: Returned explicit null for October and all three December values (4/4).
- [Fed H.4.1 · pages 1–2, Table 1](<https://www.federalreserve.gov/releases/h41/20251229/h41.pdf>). Expected: Reserve Bank credit weekly change 21,729 and Wednesday level 6,533,633. Reserve balances weekly change −3,463 and Wednesday level 2,980,514. Units are USD millions. Observed: Returned three values correctly. Wednesday Reserve Bank credit was null in JSON on all three runs (3/4 each). A separate Markdown request contained it.
- [BEA GDP · page 7, Table 1](<https://www.bea.gov/sites/default/files/2026-01/gdp3q25-updated.pdf#page=7>). Expected: Q2 and Q3 annualized changes: real GDP 3.8% and 4.4%, private investment −13.8% and 0%. Observed: Returned both Q3 values initially (2/4). Both reruns returned all four and still flagged truncation.
- [Census scan · PDF page 24, Table 15](<https://www2.census.gov/prod2/decennial/documents/10107945v1pAch02.pdf#page=24>). Expected: California 1940 rank 5 and population 6,907,387. Illinois 1960 rank 4 and population 10,081,158. Observed: No cells by URL. Uploaded with OCR, California population matched. California rank read 55. Both Illinois numbers appeared without a reliable state label (1/4 for upload).

## Page monitoring · 10-minute polling

We changed one controlled test page and held the other at 40 units. A detection counted when the event showed the right old and new stock count and revision. The times are Pacific.

| Change | Published | Check started | Event recorded | Elapsed |
| --- | --- | --- | --- | --- |
| Stock 17 → 29 units | 11:11:15 a.m. | 11:17:07 a.m. | 11:17:13 a.m. | 5 min 58 sec |
| Stock 29 → 8 units | 11:23:08 a.m. | 11:27:33 a.m. | 11:27:39 a.m. | 4 min 31 sec |
| Stock 29 → 5 → 29 | 11:18:06 a.m. (restored 11:19:07 a.m.) | Between checks | No event | 61 sec live |
| Control held at 40 | Unchanged | 11:21:27 and 11:31:13 a.m. | No event | 0 false alerts / 2 checks |

Both lasting changes were detected on the next scheduled check. Most of each delay was the wait for the next check. Once a check began, the event appeared about six seconds later.

An on-demand check on the [public Los Angeles clock](<https://www.timeanddate.com/worldclock/usa/los-angeles>) detected the time change and correctly produced no event for the unchanged date.

## How we tested

We set expected answers before scoring. A blank counted only when the source had no value for the requested field. The results cover selected samples from one day. AgentRouter handled 27 initial requests, three diagnostics and 16 reruns. The initial requests and diagnostics cost $0.216, and the reruns cost $0.240. The dedicated G2 parser, OCR and monitoring tests ran directly on Context.dev.

## Related research

[Best Tools for Collecting Company Data from Websites](<https://www.sota2.com/benchmarks/company-website-evidence>)
