Firecrawl Scrape

Usage-based pricing
ae_firecrawl_scrape

Scrape one specified URL with Firecrawl and return content in the formats you choose. The required `url` is the page to retrieve. `formats` can request markdown, HTML, raw HTML, screenshots, links, summary, change tracking, branding, JSON, a targeted query, or audio. For structured fields, use `formats: ["json"]` with `jsonOptions` containing an extraction prompt and, if needed, a JSON schema. For a targeted answer, use the query format and provide `queryOptions.prompt` and `queryOptions.mode` (`directQuote` or `freeform`). Optional settings include `waitFor` for JavaScript-rendered pages; `onlyMainContent`, `includeTags`, and `excludeTags` for content selection; `proxy`, `mobile`, and `location` for page loading; `parsers` and `pdfOptions` for PDF processing; and `screenshotOptions` for screenshot output. `redactPII`, `removeBase64Images`, `zeroDataRetention`, and `skipTlsVerification` configure privacy or request handling. `maxAge` controls acceptable cache age; set it to `0` to require a fresh scrape. With `lockdown: true`, retrieval uses only the existing cache or index and fails on a cache miss. The operation returns the selected extracted content and page metadata in a direct response; it is not an asynchronous job. Firecrawl may reuse cached content, so a successful result does not necessarily represent the page's current live state. This tool handles one known URL; use crawl for multiple pages or map/search to discover URLs.

Productivity
Usage-based pricing

Pricing

The charge per call is (max(1.2 × web scraping usage (units), 0)) AE credits.

Billing is based on the actual usage incurred when scraping the specified URL and generating the selected output formats.

Parameters

Optional parameters use the platform defaults when omitted.

ParameterTypeRequiredDescriptionDefault
urlstringYesNo parameter description.—
proxyenum (basic | stealth | enhanced | auto)NoNo parameter description.—
maxAgenumberNoMaximum acceptable age of cached content before a fresh scrape is required.—
mobilebooleanNoWhether to render the page using a mobile device configuration.—
formatsarrayNoOutput formats to generate for the scraped page, such as markdown, JSON, query results, screenshots, or branding data.—
parsersarrayNoNo parameter description.—
profileobjectNoNo parameter description.—
profile.namestringYesNo parameter description.—
profile.saveChangesbooleanNoNo parameter description.—
waitFornumberNoTime to wait in milliseconds before extraction, allowing JavaScript-rendered content to load.—
locationobjectNoGeographic and language settings used when loading the page.—
location.countrystringNoNo parameter description.—
location.languagesarrayNoPreferred languages for the page request.—
lockdownbooleanNoWhether to serve the request only from the existing cache or index without making an outbound network request. The request fails on a cache miss.—
redactPIIbooleanNoWhether to redact personally identifiable information from the returned content.—
pdfOptionsobjectNoOptions for processing PDF content.—
pdfOptions.maxPagesintegerNoMaximum number of PDF pages to process.—
excludeTagsarrayNoHTML tags to exclude from the extracted content.—
includeTagsarrayNoHTML tags to include in the extracted content.—
jsonOptionsobjectNoOptions for structured JSON extraction. Use when extracting specific fields or data that must match a schema.—
jsonOptions.promptstringNoInstructions describing the specific data to extract as JSON.—
jsonOptions.schemaobjectNoJSON Schema defining the structure of the extracted data.—
queryOptionsobjectNoOptions for returning a targeted answer from page content instead of the full page.—
queryOptions.modeenum (directQuote | freeform)YesNo parameter description.freeform
queryOptions.promptstringYesQuestion or extraction instruction for the targeted query.—
storeInCachebooleanNoWhether to store the scrape result in the cache.—
onlyMainContentbooleanNoWhether to extract only the main page content and exclude navigation, headers, footers, and other boilerplate.—
screenshotOptionsobjectNoOptions for generated page screenshots.—
screenshotOptions.qualitynumberNoImage quality setting for the generated screenshot.—
screenshotOptions.fullPagebooleanNoWhether to capture the entire scrollable page instead of only the visible viewport.—
screenshotOptions.viewportobjectNoViewport dimensions to use when rendering the screenshot.—
screenshotOptions.viewport.widthnumberYesNo parameter description.—
screenshotOptions.viewport.heightnumberYesNo parameter description.—
zeroDataRetentionbooleanNoWhether to avoid retaining data from this scrape request.—
removeBase64ImagesbooleanNoWhether to remove base64-encoded images from the extracted content.—
skipTlsVerificationbooleanNoWhether to skip TLS certificate verification when requesting the URL.—

Invocation example

Copy the request and explore its response example.

POST/agent-api/v1/tool/executeX-Api-Key
Request coderequest.sh
curl --request POST \
  --url 'https://agentearth.ai/agent-api/v1/tool/execute?tool_name=ae_firecrawl_scrape' \
  --header "X-Api-Key: $AGENTEARTH_API_KEY" \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "params": {
    "formats": [
      "json"
    ],
    "jsonOptions": {
      "prompt": "Return ONLY the raw active markets as a JSON array. For each market include exactly these original fields and nothing else: conditionId, question, outcomes, outcomePrices, clobTokenIds, endDate, bestBid, bestAsk, liquidityNum, volume24hr, negRisk, active, closed, acceptingOrders, updatedAt. Skip any market with closed=true or acceptingOrders=false. Do not summarize, do not add commentary.",
      "schema": {
        "items": {
          "properties": {
            "acceptingOrders": {
              "type": "boolean"
            },
            "active": {
              "type": "boolean"
            },
            "bestAsk": {
              "type": "number"
            },
            "bestBid": {
              "type": "number"
            },
            "clobTokenIds": {
              "type": "string"
            },
            "closed": {
              "type": "boolean"
            },
            "conditionId": {
              "type": "string"
            },
            "endDate": {
              "type": "string"
            },
            "liquidityNum": {
              "type": "number"
            },
            "negRisk": {
              "type": "boolean"
            },
            "outcomePrices": {
              "type": "string"
            },
            "outcomes": {
              "type": "string"
            },
            "question": {
              "type": "string"
            },
            "updatedAt": {
              "type": "string"
            },
            "volume24hr": {
              "type": "number"
            }
          },
          "type": "object"
        },
        "type": "array"
      }
    },
    "maxAge": 0,
    "onlyMainContent": true,
    "url": "https://gamma-api.polymarket.com/markets?closed=false&limit=100&order=volume24hr&ascending=false"
  }
}'
Response exampleresponse.json
JSONPartial content
{
  "agent_hints": [
    "This site has a structured-data provider (polymarket). To get typed records instead of page text, read its capabilities with alexandria_library_search {\"command\":\"cat /library/*/polymarket/README.md\"}, then call alexandria_library_read {\"calls\":[{\"provider"
  ],
  "conversationId": "73d45272-ab7b-4e90-989a-1da95b38c03a",
  "json": [
    {
      "acceptingOrders": true,
      "active": true,
      "bestAsk": 0.039,
      "bestBid": 0.038,
      "clobTokenIds": "[\"42865728742506860386768556888821101266630139112044622335353850919229585308601\"]",
      "closed": false
    }
  ],
  "metadata": {
    "concurrencyLimited": false,
    "contentType": "application/json",
    "indexId": "0b0d3025-2f01-4102-bf76-c2c7c432f8c6",
    "proxyUsed": "basic",
    "scrapeId": "01a1c9c2-d187-75e9-9861-f5e467ad7933",
    "url": "https://gamma-api.polymarket.com/markets?closed=false&limit=100&order=volume24hr&ascending=false"
  },
  "warning": "The extraction content would have used more tokens (155706+) than the maximum we allow (102400). -- the input has been automatically trimmed."
}

This example shows part of the response. Copied content is valid JSON.

Guides using this tool

See how this tool fits into real workflows