Context.dev API Feature Review

We tested what Context.dev could extract from known webpage URLs. We also tested its dedicated G2 review parser, PDF table extraction with and without OCR, and scheduled page monitoring.

Tested 27 initial requests16 reruns

Results at a glance

Each row has its own unit: fields, storefronts, reviews, table cells or change events. Text PDFs show first-run results. The monitor row counts changes that stayed visible through a scheduled check.

Public webpages First attempt

We score each field separately, such as a price, room size or stock status. For each App Store page, we check whether the app is listed or absent in that country. A page counts as correct only if every answer matches the source.

First attempts on public webpages
TaskPages with every answer correctField checks passed
Software pricing4/4
13/13
Hotel room details3/4
7/11
Store prices and stock1/3
5/8
App Store visibility8/8
8/8

Software pricing passed on all 4 pages, with 13 of 13 pricing fields matched. Allbirds matched 3 of 4 fields, so that page did not pass. The API left size 8 stock blank.

See all 19 webpages, expected answers and first responses

We used JSON extraction for pricing, hotels and stores. For App Store URLs we used Markdown retrieval. We checked the storefront against Apple’s page and its country-specific app lookup. Both had to confirm an absent listing for an HTTP 404 to count as correct.

Software pricing

Software pricing first-attempt evidence
Input sourceWhat we checked and expectedWhat came back
MomenticStandard step $0.0125. Browser per minute $0.005 Standard, $0.0067 Large.All 3 values matched.
LinearBasic $10 and Business $16 per user per month. Business billed annually. Enterprise custom pricing, so no number.All 4 values matched, including an explicit null for Enterprise.
ResendPro $20/month, 50,000 included emails/month and $0.90 per 1,000 extra emails.All 3 values matched.
PostHog1 Free project, 6 pay-as-you-go projects and 2 years of pay-as-you-go retention.All 3 values matched.

Hotel room details

Hotel room details first-attempt evidence
Input sourceWhat we checked and expectedWhat came back
Park Hyatt Kuala LumpurMinimum areas of 54 m² for Tower King and 104 m² for Park Suite, plus the bedroom photo URL for each.Page access error. No requested values returned (0/4).
The Singapore EDITIONGarden Premier King photo and Premier Suite living room photo, matched by gallery caption and image asset.Both image assets matched (2/2).
The Hoxton, ShoreditchShoebox 12 m², Cosy Up 20 m², and Cosy Up maximum occupancy of 2 adults.All 3 values matched.
The LudlowLoft King 325 ft² and Ludlow Loft 405 ft².Both areas matched (2/2).

Store prices and stock

Store prices and stock first-attempt evidence
Input sourceWhat we checked and expectedWhat came back
Allbirds MistSizes 8 and 9 both $100. Size 8 out of stock, size 9 in stock.Both prices and size 9 stock matched. Size 8 stock returned null (3/4).
Ugmonk Walnut base kit$69 and in stock.Both values matched (2/2).
Ugmonk Walnut kit with refills$89 and in stock.Page access error. No requested values returned (0/2).

App Store visibility

App Store visibility first-attempt evidence
Input sourceWhat we checked and expectedWhat came back
Spotify · USPublic listing visible.Listing returned for the US storefront.
Spotify · UKPublic listing visible.Listing returned for the UK storefront.
Venmo · USPublic listing visible.Listing returned for the US storefront.
Venmo · UKNo public listing.Source returned HTTP 404.
BBC iPlayer · UKPublic listing visible.Listing returned for the UK storefront.
BBC iPlayer · USNo public listing.Source returned HTTP 404.
PayPay · JapanPublic listing visible.Listing returned for the Japan storefront.
PayPay · USNo public listing.Source returned HTTP 404.

Same request, two reruns

We sent the same requests through AgentRouter for the first run and both reruns. Three of the five samples with initial misses returned every field correctly on both reruns. Each bar shows how many fields matched on that attempt.

Fields matched across the first run and two reruns. Bar height shows the share of fields matched for each sample.
Test sampleFirst runRerun 1Rerun 2
Park Hyatt Kuala LumpurBoth room sizes matched. The API returned no photo links.
0/4
2/4
2/4
Ugmonk kit with refillsPage access error first. Both reruns found $89 and in stock.
0/2
2/2
2/2
Allbirds MistSize 8 stock was missing initially. Both reruns found it out of stock.
3/4
4/4
4/4
Federal Reserve H.4.1No Wednesday Reserve Bank credit value on any of the three attempts.
3/4
3/4
3/4
BEA GDPBoth reruns found the missing Q2 GDP (3.8%) and investment (−13.8%) figures.
2/4
4/4
4/4

Both reruns scored the same on all five samples. Independent checks found the same merchant reference values and identical Fed and BEA PDF files. The scored calls show how the full route behaved. They cannot pinpoint whether a difference came from AgentRouter, Context.dev’s API or access to the source.

How to isolate the cause of a changed result

Send the same URL, schema and options directly to Context.dev and through AgentRouter close together, several times, using a fixed source. For each gateway call, compare the raw upstream response with what the gateway returned. A field present upstream and missing from that same call's tool output points to the gateway. If direct API responses vary on the fixed source, the upstream path can vary on its own.

We logged gateway queue rejections separately. Those requests were rejected before execution and are outside the scores shown here.

See the Hyatt response and the three passing controls

The Hyatt request asked for the floor area and bedroom photo URL of the Tower King and Park Suite. The first request returned a page access error. Both reruns returned 54 m² and 104 m² and an explicit null for each photo URL, so the answer stayed at 2/4. The reference photos were listed on the source.

We also resent three requests that had already passed. The Singapore EDITION photos matched 2/2, Ugmonk’s base kit price and stock matched 2/2, and the BLS CPI table matched 4/4 on every attempt. Their original inputs and expected answers are listed under Public webpages and PDF tables and OCR.

G2 reviews IDs, ratings and dates

We gave the G2 parser two product review-list URLs. We matched each returned review to the public listings by ID, then compared its star rating and publication date. Both products had fewer than 25 public reviews.

Native G2 parser results by product
G2 productID and rating matchedDate matched (Pacific)
Plausible Analytics
7/7
6/7
Umami
6/6
4/5

All 13 IDs and ratings matched. 10 of 12 publication dates matched the public display after converting the API timestamps to Pacific time. One Umami review showed only its last edit date. It counted for ID and rating only.

See the 13 review records and four named lookups

Eight of 12 display dates matched in UTC. Pacific time resolved two one-day differences. A Plausible review still differed by 14 days and an Umami review by four days. The stars below are the same on G2 and in the API response.

Plausible Analytics

Plausible Analytics review-level results
Review IDStarsDate on G2API date (Pacific)
1405929652026-11-112026-11-11
1387145652026-10-272026-10-27
1385081852026-10-232026-10-23
1362858552026-09-282026-09-28
97653554.52024-06-222024-06-08
843520852023-08-092023-08-09
683377752022-07-102022-07-10

Umami

Umami review-level results
Review IDStarsDate on G2API date (Pacific)
123710284.52026-02-172026-02-17
100324464.52024-08-212024-08-21
99282324.52024-07-262024-07-26
992697052024-07-262024-07-26
71845553.5Updated 2023-03-282022-10-10
759749552023-01-192023-01-15

Earlier test of four named reviews

Each request named a review and asked for its rating and date. Neither route got a complete answer (0/4 each). The G2 parser returns up to 25 recent reviews per product and has no pagination. G2 parser documentation

Earlier named-review requests and responses
Input sourceWhat we checked and expectedWhat came back
Slack“Intuitive Team Collaboration, Though Search and Alerts Need Work.” 4.5 stars, published 2026-11-04, updated 2026-11-08.Generic scraper: page access error. G2 parser: target absent from 25 results.
monday CRM“monday CRM with Unleashed made our sales process smooth.” 4.5 stars, published 2026-11-09.Generic scraper: page access error. G2 parser: target absent from 25 results.
Linear“Linear hits the right balance of performance and features.” 5 stars, published 2026-08-10.Generic scraper: page access error. G2 parser: target absent from 25 results.
Plausible Analytics“easy to use and secure tool for site tracking.” 4.5 stars, published 2024-06-22.Generic scraper: both fields null. G2 parser: found the target among 7 results, returned 4.5 stars and 2024-06-08.

PDF tables and OCR

We asked for four specific table cells per PDF. To count as correct, each value had to come from the requested row and column, with the right period and unit.

Text PDFs

9/12

Correct cells across three text PDFs on the first run. The Fed PDF was still missing the same cell after both reruns.

Uploaded Census scan with OCR

1/4

We used optical character recognition to read four table cells from one scanned page. Only California’s 1940 population was correct. OCR read California’s rank as 55 instead of 5. It read both Illinois numbers without keeping them tied to the Illinois row.

See the four PDFs, exact cells and responses

We sent the public PDF URLs to the scraper and asked for structured JSON. The first Census scan request had no OCR and found no cells. Enabling OCR by URL returned an access error. We then uploaded just PDF page 24 with OCR enabled.

PDF inputs, requested cells and observed results
Input sourceWhat we checked and expectedWhat came back
BLS CPI · page 2, Table AOctober all-items monthly change unpublished. December monthly change 0.3%, annual change 2.7%, gasoline annual change −3.4%.Returned explicit null for October and all three December values (4/4).
Fed H.4.1 · pages 1–2, Table 1Reserve Bank credit weekly change 21,729 and Wednesday level 6,533,633. Reserve balances weekly change −3,463 and Wednesday level 2,980,514. Units are USD millions.Returned three values correctly. Wednesday Reserve Bank credit was null in JSON on all three runs (3/4 each). A separate Markdown request contained it.
BEA GDP · page 7, Table 1Q2 and Q3 annualized changes: real GDP 3.8% and 4.4%, private investment −13.8% and 0%.Returned both Q3 values initially (2/4). Both reruns returned all four and still flagged truncation.
Census scan · PDF page 24, Table 15California 1940 rank 5 and population 6,907,387. Illinois 1960 rank 4 and population 10,081,158.No cells by URL. Uploaded with OCR, California population matched. California rank read 55. Both Illinois numbers appeared without a reliable state label (1/4 for upload).

PDF scraping docs · OCR docs

Page monitoring 10-minute polling

We set a 10-minute check schedule for two test pages. We changed the stock count on one page and held the other at 40 units. The chart measures the time from publishing each change to Context.dev recording a change event.

  • Stock 17 → 29 units5 min 58 sec
  • Stock 29 → 8 units4 min 31 sec
No eventStock 29 → 5 → 29 reverted after 61 seconds, between polls.

Both lasting changes were detected on the next scheduled check (2/2). Most of each delay was the wait for that check. Once a check began, the event appeared about six seconds later in both cases. The unchanged page produced zero events in two checks.

This schedule can catch changes that stay visible until the next check. A brief change can come and go between checks, as the 61-second update did.

See the test page inputs, exact times and events

We recorded the baseline text on two static pages we hosted for the test: Stock DEMO-A | revision 0 | 17 units available and Stock DEMO-B | revision 0 | 40 units available. We verified each update on the public URL. A detection counted if the event showed the right old and new text, including the revision and stock count. All times are Pacific. Monitor schedules

Controlled webpage updates and events
InputPublished or checkedWhat came back
Stock 17 → 29Published 11:11:15 a.m. Check began 11:17:07 a.m.Event at 11:17:13 a.m. with the correct old and new text.
Stock 29 → 5 → 29Published 11:18:06 a.m., restored 11:19:07 a.m.No event. The update came and went between two checks.
Stock 29 → 8Published 11:23:08 a.m. Check began 11:27:33 a.m.Event at 11:27:39 a.m. with the correct old and new text.
Control held at 40Automatic checks started at 11:21:27 and 11:31:13 a.m.No change and no event ID on either check.

We also tested Run now on the public Los Angeles clock. The clock element changed from 11:01:50 to 11:02:28 a.m. and produced a change event. The date element stayed on November 16 and produced no event (2/2 expected outcomes).

What to use it for

  • Pricing and storefrontsStart with known URLs and explicit fields. Recheck blank stock values.
  • Recent G2 reviewsMatch on review IDs and verify dates that drive a decision.
  • PDF tables and alertsCheck required cells against the source. Polling can miss changes shorter than the interval.

How we tested

We set the expected answers before scoring. An explicit empty value counted as correct only when the source also had no value for that field. The results cover selected samples from one day.

Requests, timing and cost

AgentRouter handled 27 initial requests, three diagnostics and 16 reruns. The initial requests and diagnostics cost $0.216 and the reruns cost $0.240. We ran the original requests and scoring rules again in two windows about 15 minutes apart. Native G2, OCR and monitoring ran directly on Context.dev.

Merchant reference data matched before and after testing. The Fed and BEA PDFs had identical hashes across rerun checks. Direct Hyatt and BLS reference fetches returned HTTP 403 during reruns, so we checked archived values against indexed copies of their official pages.

Sources

SOTA2, Context.dev API Feature Review, November 16, 2026.