Watches

An agent can GET a page. What it cannot do is keep watching one — stay resident, poll it every few minutes for months, and wake up only when something it cares about changes. A watch is that missing half. You hand us a source and a rule for what counts as "interesting"; we poll it on a schedule, diff each result against the last, and fire a webhook the moment your rule matches. A watch turns pull-once into push-on-change — the agent stops babysitting a URL and gets told instead.

You point a watch at a page, an RSS feed, or a JSON endpoint, give it a match_rule and a webhook_url, and choose how often to poll. We do the boring part — fetching politely, holding the diff state, coalescing your poll with every other Team watching the same source — and you get one clean event per change that matters.

How watches behave

A watch has three moving parts: where to look (a source), what counts as a change worth telling you about (a match_rule), and how often to check (a poll_interval).

Source types. A watch names one of three source_types, and the extraction differs per type:

Match rules. A match_rule is a small JSON object with a kind — there are exactly three, with no boolean composition of them:

• {"kind": "any"} — fire on every change. The default.
• {"kind": "regex", "pattern": "..."} — fire when the new content matches a regular expression. Patterns run on a linear-time engine (re2), so catastrophic backtracking is impossible; backreferences and lookaround are rejected at create time.
• {"kind": "jsonpath", "path": "$.price", "op": "lt", "value": 100} — for a json_endpoint, fire when the value at a JSONPath satisfies a comparison. The op is one of eq, ne, lt, lte, gt, gte, contains, or changed. (Plain equality is the eq op here, not its own kind.)

Statuses. A Watch.status is either active or paused — that is the whole set. active watches are polled and are the stock the tier limit counts; paused watches keep all their config but stop being polled and drop out of the count. There is deliberately no error status on a watch — see When a source errors.

Every change is recorded, matched or not. A watch keeps a history of the changes it detected — and it records a change even when your match_rule didn't match it. Only matched changes fire a webhook, but the unmatched ones are still logged (with no delivery), so the history can tell you a source "changed 12 times, matched twice". That gap is the signal you need to loosen a rule that is too narrow.

Polling etiquette and private sources

We poll other people's servers on your behalf, so we do it politely. We honour robots.txt and Cache-Control, we identify ourselves with a stable User-Agent, and a shared per-domain politeness budget caps how fast all of our watches hit any one host — no Team can turn us into a battering ram against a source. A poll that would breach the budget is simply slipped to a later cycle; your watch never loses a change, it just learns about it a little later.

Private sources. Many things worth watching sit behind an API key. A watch can carry a flat headers map — {"Authorization": "Bearer ..."} — that we send verbatim on every poll. Those header values are a credential, so they are stored encrypted at rest, live on your per-Team watch (never on the shared source), and are write-only: no read surface — API, MCP, or dashboard — ever echoes them back. (User-Agent is reserved and cannot be set; it identifies us.)

The ignore_robots attestation. Some private endpoints you are entitled to poll still ship a blanket robots.txt that disallows crawlers. For those, a watch may set ignore_robots: true — an attestation that you are authorised to poll this source, which skips the robots.txt gate. It is deliberately fenced:

• It requires auth headers. A header-less ignore_robots is a hard reject at create time — the attestation only makes sense for a source you hold a credential for.
• It overrides robots.txt alone. It never relaxes the SSRF guard (a watch can still only reach public hosts) and never lifts the per-domain politeness budget. Those are safety and fairness limits, not crawler etiquette.
• It is immutable. The robots posture is baked into the source's identity at create time; changing it is a delete-and-recreate, never an edit. The choice is recorded in the watch.created audit row, attributed to your Team, the key or user, and the time.

Create a watch

Creating a watch registers your match_rule, webhook_url, headers and requested poll_interval against a source. Behind the scenes we coalesce: if another Team already watches the same URL with the same extraction config, your watch rides the same physical poll rather than doubling the load on the origin. Coalescing is invisible to you — you still get your own history and your own events — but it is why watching a popular page costs the source no more than one visitor. Creation is idempotent: pass an idempotency key and a retried request returns the original watch instead of registering a second one (see Idempotency).

# A url watch with a regex rule.
curl https://aigears.example.com/v1/watches/ \
  -H "Authorization: Bearer aigears_live_..." \
  -H "Idempotency-Key: pricing-page-watch-1" \
  -H "Content-Type: application/json" \
  -d '{
    "source_type": "url",
    "url": "https://example.com/pricing",
    "css_selector": "#price",
    "match_rule": {"kind": "regex", "pattern": "\\$[0-9]+"},
    "webhook_url": "https://hooks.example.com/aigears",
    "poll_interval": "PT15M"
  }'

# An rss watch — fire on any new item.
curl https://aigears.example.com/v1/watches/ \
  -H "Authorization: Bearer aigears_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "source_type": "rss",
    "url": "https://example.com/blog/feed.xml",
    "match_rule": {"kind": "any"},
    "webhook_url": "https://hooks.example.com/aigears",
    "poll_interval": "PT1H"
  }'

# A json_endpoint watch — fire when a JSONPath value drops below a threshold.
curl https://aigears.example.com/v1/watches/ \
  -H "Authorization: Bearer aigears_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "source_type": "json_endpoint",
    "url": "https://api.example.com/v1/inventory",
    "jsonpath": "$.items[0].stock",
    "match_rule": {"kind": "jsonpath", "path": "$.items[0].stock", "op": "lt", "value": 10},
    "webhook_url": "https://hooks.example.com/aigears",
    "poll_interval": "PT30M"
  }'

Pause, resume, delete

These are the three levers on a live watch. Pause stops the polling but keeps every setting — the source, the rule, the headers — so you can resume later and pick up exactly where you left off. A paused watch drops out of your active_watches count, so pausing is also how you stop paying for a watch you want to keep on the shelf. Delete removes it for good. Pause and resume are no-ops-with-a-409 if the watch is already in the target state; delete of an already-gone watch under the same idempotency key is a quiet success, not a 404.

curl -X POST https://aigears.example.com/v1/watches/01J9Z8XKQR7M4T2VN0B3CE5DHF/pause/ \
  -H "Authorization: Bearer aigears_live_..."

Review change history

Every change a watch detects is a stored WatchEvent — the change's timestamp, its content hash, a bounded excerpt (for matched changes), and whether it matched your rule. This is the log that shows "changed 12 times, matched twice". One thing to know up front: the change history is read over MCP and the dashboard, not the REST API. The REST surface lists and fetches watches (their current status and config); the per-change WatchEvent log is not a REST endpoint in v1. If your agent needs the history programmatically, that is what the MCP watch_history tool is for.

curl "https://aigears.example.com/v1/watches/?status=active" \
  -H "Authorization: Bearer aigears_live_..."

The change webhook

When a poll finds a change your rule matched, we fire one watch.changed webhook. Its payload is new-only and bounded — and that shape is a deliberate privacy and cost decision, not a limitation to work around. We store only a hash of each source's content, never the content itself, so the old value literally does not exist for us to send; and the new value is a bounded excerpt (the changed region plus a capped prefix), never the whole body — we will not POST a 5 MB page to your endpoint. If you need to diff against the previous value, keep it from the last event you received.

Inside the standard signed envelope, the data identifies the watch and carries the new excerpt:

{
  "event_type": "watch.changed",
  "data": {
    "watch_id": "01J9Z8XKQR7M4T2VN0B3CE5DHF",
    "source_type": "url",
    "content_hash": "9f2b...",
    "changed": true,
    "excerpt": "Pro plan — $79 / month",
    "match": {"kind": "regex", "pattern": "\\$[0-9]+"},
    "detected_at": "2026-07-03T14:05:11Z"
  }
}

For an rss source, "change" means "a feed item we haven't seen before", so a feed that adds three items between polls produces one watch.changed event per new item — never a single lumped event. The envelope, the X-AIGears-Signature header, and the retry/dead-letter rules are the same for every primitive and are documented once in Webhooks & signing; for standing up a receiver and verifying the signature, see Receiving webhooks. Delivery is at-least-once, so make your handler idempotent on event_id.

When a source errors

Sources go down, move, or start returning errors. When that happens the failure belongs to the source, not to your watch: there is no watch-level error status. A watch reads its health through to the shared source's error_state, which moves from healthy to a transient backoff (we keep retrying with growing gaps) to errored (a source failing for 24 hours or more, probed hourly) — or to robots_blocked if a source that was fine at create time later disallows us.

Crucially, an erroring source does not pause or unbill your watch. The watch stays active and stays billable — a source we can't reach is not a reason for us to silently stop charging for a watch you asked us to keep. We do tell you: the source's error state shows on the watch's dashboard detail page, and we fire one watch.source_error webhook per subscribed Team (carrying the error_state and when it began) so an agent can react. Pause and delete are your only levers — if a source is dead for good, pause the watch to stop paying, or delete it.

Limits

Watches are metered as a stock: what counts is how many watches are active at once, tracked as your tier's active_watches limit. Pausing a watch takes it out of the count (and delete removes it entirely), so the count is your live watches, not every watch you ever created. Reaching the cap makes the next create return 429.

Two tier settings shape watches. active_watches caps how many you can run at once. poll_interval_floor sets the fastest you may poll — a request for a tighter interval than your tier allows is clamped up to the floor, never rejected, so a watch always polls at least as slow as the floor. The per-tier numbers, and what happens when you reach a cap, live in Limits & tiers.

Related examples

• Watch the pricing page
• The renewals agent