Ae · Firecrawl · Scrape

1.00积分/次

ae_firecrawl_scrape

读取指定网页，可返回Markdown或通过formats=["json"]及jsonOptions进行结构化抽取。替代已弃用的extract进行已知网址抽取。目标 URL 应无需登录即可公开访问；私有页面、受限社交群组和需要会话权限的页面可能无法抓取。公开页面如有反爬限制，也可能无法提取。

效率工具

1.00积分/次

价格说明

1.00积分/次

API 调用

通过统一工具网关发起同步请求，鉴权和计费由 AgentEarth 处理。

cURL

curl --request POST \
  --url 'https://api.agentearth.ai/v1/tools/run' \
  --header 'Authorization: Bearer $AGENTEARTH_API_KEY' \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "tool": "ae_firecrawl_scrape",
  "input": {
    "url": "xxx",
    "proxy": "xxx",
    "maxAge": "xxx",
    "mobile": "xxx",
    "formats": "xxx",
    "parsers": "xxx",
    "profile": "xxx",
    "waitFor": "xxx",
    "location": "xxx",
    "lockdown": "xxx",
    "redactPII": "xxx",
    "pdfOptions": "xxx",
    "excludeTags": "xxx",
    "includeTags": "xxx",
    "jsonOptions": "xxx",
    "queryOptions": "xxx",
    "storeInCache": "xxx",
    "onlyMainContent": "xxx",
    "screenshotOptions": "xxx",
    "zeroDataRetention": "xxx",
    "removeBase64Images": "xxx",
    "skipTlsVerification": "xxx"
  }
}'

请求方式: POST
接口地址: /v1/tools/run
鉴权方式: Bearer Token
超时时间: 60 秒
当前版本: —

响应示例

{
  "error_no": 0,
  "error_msg": "",
  "result": [
    {
      "type": "text",
      "text": "xxx"
    }
  ],
  "platform_annotations": {
    "source": "AgentEarth",
    "platform_note": "xxx",
    "platform_suggestion": "xxx"
  }
}

工具参数

未传入的可选参数将使用平台默认值。

参数	类型	必填	说明	默认值
`url`	string	是	The URL of the page to scrape. Target URLs should be publicly accessible without login. Private pages, restricted social groups, and pages requiring session permissions may not be retrievable. Public accessibility does not guarantee successful extraction where the site blocks automated access.	—
`proxy`	enum (basic \| stealth \| enhanced \| auto)	否	Proxy mode used to retrieve the page.	—
`maxAge`	number	否	Maximum acceptable age of cached content before a fresh scrape is required.	—
`mobile`	boolean	否	Whether to render the page using a mobile device configuration.	—
`formats`	array	否	Output formats to generate for the scraped page, such as markdown, JSON, query results, screenshots, or branding data.	—
`parsers`	array	否	Optional content parsers. Use pdf for PDF documents.	—
`profile`	object	否	Browser profile configuration used while scraping.	—
`waitFor`	number	否	Time to wait in milliseconds before extraction, allowing JavaScript-rendered content to load.	—
`location`	object	否	Geographic and language settings used when loading the page.	—
`lockdown`	boolean	否	Whether to serve the request only from the existing cache or index without making an outbound network request. The request fails on a cache miss.	—
`redactPII`	boolean	否	Whether to redact personally identifiable information from the returned content.	—
`pdfOptions`	object	否	Options for processing PDF content.	—
`excludeTags`	array	否	HTML tags to exclude from the extracted content.	—
`includeTags`	array	否	HTML tags to include in the extracted content.	—
`jsonOptions`	object	否	Options for structured JSON extraction. Use when extracting specific fields or data that must match a schema.	—
`queryOptions`	object	否	Options for returning a targeted answer from page content instead of the full page.	—
`storeInCache`	boolean	否	Whether to store the scrape result in the cache.	—
`onlyMainContent`	boolean	否	Whether to extract only the main page content and exclude navigation, headers, footers, and other boilerplate.	—
`screenshotOptions`	object	否	Options for generated page screenshots.	—
`zeroDataRetention`	boolean	否	Whether to avoid retaining data from this scrape request.	—
`removeBase64Images`	boolean	否	Whether to remove base64-encoded images from the extracted content.	—
`skipTlsVerification`	boolean	否	Whether to skip TLS certificate verification when requesting the URL.	—