Firecrawl Scrape
Scrape one specified URL with Firecrawl and return content in the formats you choose. The required `url` is the page to retrieve. `formats` can request markdown, HTML, raw HTML, screenshots, links, summary, change tracking, branding, JSON, a targeted query, or audio. For structured fields, use `formats: ["json"]` with `jsonOptions` containing an extraction prompt and, if needed, a JSON schema. For a targeted answer, use the query format and provide `queryOptions.prompt` and `queryOptions.mode` (`directQuote` or `freeform`). Optional settings include `waitFor` for JavaScript-rendered pages; `onlyMainContent`, `includeTags`, and `excludeTags` for content selection; `proxy`, `mobile`, and `location` for page loading; `parsers` and `pdfOptions` for PDF processing; and `screenshotOptions` for screenshot output. `redactPII`, `removeBase64Images`, `zeroDataRetention`, and `skipTlsVerification` configure privacy or request handling. `maxAge` controls acceptable cache age; set it to `0` to require a fresh scrape. With `lockdown: true`, retrieval uses only the existing cache or index and fails on a cache miss. The operation returns the selected extracted content and page metadata in a direct response; it is not an asynchronous job. Firecrawl may reuse cached content, so a successful result does not necessarily represent the page's current live state. This tool handles one known URL; use crawl for multiple pages or map/search to discover URLs.
Pricing
The charge per call is (max(1.2 × web scraping usage (units), 0)) AE credits.
Billing is based on the actual usage incurred when scraping the specified URL and generating the selected output formats.
Parameters
Optional parameters use the platform defaults when omitted.
| Parameter | Type | Required | Description | Default |
|---|---|---|---|---|
url | string | Yes | No parameter description. | — |
proxy | enum (basic | stealth | enhanced | auto) | No | No parameter description. | — |
maxAge | number | No | Maximum acceptable age of cached content before a fresh scrape is required. | — |
mobile | boolean | No | Whether to render the page using a mobile device configuration. | — |
formats | array | No | Output formats to generate for the scraped page, such as markdown, JSON, query results, screenshots, or branding data. | — |
parsers | array | No | No parameter description. | — |
profile | object | No | No parameter description. | — |
profile.name | string | Yes | No parameter description. | — |
profile.saveChanges | boolean | No | No parameter description. | — |
waitFor | number | No | Time to wait in milliseconds before extraction, allowing JavaScript-rendered content to load. | — |
location | object | No | Geographic and language settings used when loading the page. | — |
location.country | string | No | No parameter description. | — |
location.languages | array | No | Preferred languages for the page request. | — |
lockdown | boolean | No | Whether to serve the request only from the existing cache or index without making an outbound network request. The request fails on a cache miss. | — |
redactPII | boolean | No | Whether to redact personally identifiable information from the returned content. | — |
pdfOptions | object | No | Options for processing PDF content. | — |
pdfOptions.maxPages | integer | No | Maximum number of PDF pages to process. | — |
excludeTags | array | No | HTML tags to exclude from the extracted content. | — |
includeTags | array | No | HTML tags to include in the extracted content. | — |
jsonOptions | object | No | Options for structured JSON extraction. Use when extracting specific fields or data that must match a schema. | — |
jsonOptions.prompt | string | No | Instructions describing the specific data to extract as JSON. | — |
jsonOptions.schema | object | No | JSON Schema defining the structure of the extracted data. | — |
queryOptions | object | No | Options for returning a targeted answer from page content instead of the full page. | — |
queryOptions.mode | enum (directQuote | freeform) | Yes | No parameter description. | freeform |
queryOptions.prompt | string | Yes | Question or extraction instruction for the targeted query. | — |
storeInCache | boolean | No | Whether to store the scrape result in the cache. | — |
onlyMainContent | boolean | No | Whether to extract only the main page content and exclude navigation, headers, footers, and other boilerplate. | — |
screenshotOptions | object | No | Options for generated page screenshots. | — |
screenshotOptions.quality | number | No | Image quality setting for the generated screenshot. | — |
screenshotOptions.fullPage | boolean | No | Whether to capture the entire scrollable page instead of only the visible viewport. | — |
screenshotOptions.viewport | object | No | Viewport dimensions to use when rendering the screenshot. | — |
screenshotOptions.viewport.width | number | Yes | No parameter description. | — |
screenshotOptions.viewport.height | number | Yes | No parameter description. | — |
zeroDataRetention | boolean | No | Whether to avoid retaining data from this scrape request. | — |
removeBase64Images | boolean | No | Whether to remove base64-encoded images from the extracted content. | — |
skipTlsVerification | boolean | No | Whether to skip TLS certificate verification when requesting the URL. | — |
Invocation example
Copy the request and explore its response example.
/agent-api/v1/tool/executeX-Api-Keycurl --request POST \
--url 'https://agentearth.ai/agent-api/v1/tool/execute?tool_name=ae_firecrawl_scrape' \
--header "X-Api-Key: $AGENTEARTH_API_KEY" \
--header 'Content-Type: application/json' \
--data-raw '{
"params": {
"formats": [
"json"
],
"jsonOptions": {
"prompt": "Return ONLY the raw active markets as a JSON array. For each market include exactly these original fields and nothing else: conditionId, question, outcomes, outcomePrices, clobTokenIds, endDate, bestBid, bestAsk, liquidityNum, volume24hr, negRisk, active, closed, acceptingOrders, updatedAt. Skip any market with closed=true or acceptingOrders=false. Do not summarize, do not add commentary.",
"schema": {
"items": {
"properties": {
"acceptingOrders": {
"type": "boolean"
},
"active": {
"type": "boolean"
},
"bestAsk": {
"type": "number"
},
"bestBid": {
"type": "number"
},
"clobTokenIds": {
"type": "string"
},
"closed": {
"type": "boolean"
},
"conditionId": {
"type": "string"
},
"endDate": {
"type": "string"
},
"liquidityNum": {
"type": "number"
},
"negRisk": {
"type": "boolean"
},
"outcomePrices": {
"type": "string"
},
"outcomes": {
"type": "string"
},
"question": {
"type": "string"
},
"updatedAt": {
"type": "string"
},
"volume24hr": {
"type": "number"
}
},
"type": "object"
},
"type": "array"
}
},
"maxAge": 0,
"onlyMainContent": true,
"url": "https://gamma-api.polymarket.com/markets?closed=false&limit=100&order=volume24hr&ascending=false"
}
}'{
"agent_hints": [
"This site has a structured-data provider (polymarket). To get typed records instead of page text, read its capabilities with alexandria_library_search {\"command\":\"cat /library/*/polymarket/README.md\"}, then call alexandria_library_read {\"calls\":[{\"provider"
],
"conversationId": "73d45272-ab7b-4e90-989a-1da95b38c03a",
"json": [
{
"acceptingOrders": true,
"active": true,
"bestAsk": 0.039,
"bestBid": 0.038,
"clobTokenIds": "[\"42865728742506860386768556888821101266630139112044622335353850919229585308601\"]",
"closed": false
}
],
"metadata": {
"concurrencyLimited": false,
"contentType": "application/json",
"indexId": "0b0d3025-2f01-4102-bf76-c2c7c432f8c6",
"proxyUsed": "basic",
"scrapeId": "01a1c9c2-d187-75e9-9861-f5e467ad7933",
"url": "https://gamma-api.polymarket.com/markets?closed=false&limit=100&order=volume24hr&ascending=false"
},
"warning": "The extraction content would have used more tokens (155706+) than the maximum we allow (102400). -- the input has been automatically trimmed."
}This example shows part of the response. Copied content is valid JSON.
Guides using this tool
See how this tool fits into real workflows

