Skip to content
DocsReachWeb crawler and browser

Web research

Our crawler, across your workspace.

Paste a public URL in the web reader, ask about that URL in a connected chat, call the MCP tool or use the CLI. These surfaces use the same PersistMemory crawler. Public reading and extraction make no AI model or paid crawler call.

Open web readerOpen isolated browserMonitor API documentation

Read a website

  1. Open Web scraper in the dashboard and paste the public HTTP(S) URL.
  2. Choose one page or a small crawl of up to three same-site pages. Enable Check the latest content to validate the website now.
  3. Under Reading options, choose automatic rendering, HTTP only, or always use the isolated browser. Optional CSS selectors include or exclude content; enter one selector per line, up to eight per list.
  4. Review the title, source, reading method and timestamps. Expand the text, download it or save it to an editable Space. Saving starts memory extraction and may use AI.
pm web read https://example.com/docs --fresh
pm web read https://example.com/docs --render always --include main --exclude .sidebar

The API is POST /api/v1/web/read with { url, maxPages?, fresh?, render?, includeSelectors?, excludeSelectors? }. The authenticated account owns its cache. MCP and connected chat use read_public_page with the same options. In chat, supply the exact URL in your current message.

Freshness and change detection

Ordinary reading may reuse content for five minutes. Fresh reading contacts the website; a conditional request can reuse content only after the site confirms it is unchanged. fetchedAt records when the content was obtained; checkedAt records validation, and revalidated identifies a conditional confirmation.

API watches always request fresh validation when created or checked manually or on schedule. They compare the first page's normalized extracted content with the reviewed baseline. Main content is preferred, but unusual sites can still include navigation. One reading covers at most 12,000 characters; a truncation notice means changes beyond the limit are not monitored.

Interact with a website

Open the isolated browser from the web reader. Start a session or paste an existing session ID from chat, MCP or the CLI. Inspect its screenshot, visible text and controls. Choose an observed control to click, fill or press a supported key. Navigation, extraction and screenshots are also available. Each action uses the observed version; a conflict or interrupted request requires inspection before continuing.

The MCP tools are start_browser, inspect_browser, act_in_browser and close_browser. Browser mutations require write access. Connected chat proposes an action before executing it under the existing confirmation flow. A proposal is not a completed website action.

pm web browser start https://example.com
pm web browser get <session-id>
pm web browser click <session-id> --version <observed-version> --element <observed-element-id>
pm web browser screenshot <session-id> --version <observed-version>
pm web browser close <session-id>

Passwords and other sensitive fields belong in the explicit manual dashboard input, never chat or CLI arguments. Browser state is temporary and account scoped. Closing or expiry removes its session state. Screenshots can be inspected in the dashboard; they are not inserted as large image payloads in model tool text.

Access and limits

Public reading respects robots.txt and rejects private network addresses, credential-bearing URLs and unsupported ports. JavaScript rendering does not bypass CAPTCHA or guarantee access to sites that block automation. Blocked or empty pages are reported as unavailable evidence; a failed watch check preserves its successful baseline.

A direct URL uses our crawler first. Topic search is a separate discovery capability: the deployment's existing TinyFish, Firecrawl and Brave search adapters may consume their provider quotas. A supplied URL task does not silently invoke those adapters unless the person explicitly requests web search. Search results and website text remain untrusted source material.

Configure the isolated browser

The operator deploys the browser service separately from the API and worker. Configure both CRAWLER_RENDERER_URL and CRAWLER_RENDERER_TOKEN for the API, MCP and worker; the URL identifies your own service and the token authenticates its control plane. Do not expose this token in client code or a chat.

Without the browser service, public HTTP extraction remains available. Explicit browser sessions and rendering requests report that rendering is not configured. Deployment and browser infrastructure can incur hosting costs even though reading uses no paid crawler provider.