Dumpling AI Crawl Website

7.5 credits/call
ae_dumplingai_crawl_website

Crawl multiple pages of a website to capture content and metadata using DumplingAI.

Productivity
7.5 credits/call

Pricing

7.5 credits/call
ConditionCharge
Each call7.5 credits/call

Parameters

Optional parameters use the platform defaults when omitted.

ParameterTypeRequiredDescriptionDefault
urlstringYesRoot URL to crawl.—
depthintegerNoMaximum crawl depth.2
limitintegerNoMaximum number of pages to fetch.5
formatenum (markdown | text | raw)NoOutput format for the crawled pages.markdown
requestSourcestringNoOptional request source identifier.—

Invocation example

Copy the request and explore its response example.

POST/agent-api/v1/tool/executeX-Api-Key
Request coderequest.sh
curl --request POST \
  --url 'https://agentearth.ai/agent-api/v1/tool/execute?tool_name=ae_dumplingai_crawl_website' \
  --header "X-Api-Key: $AGENTEARTH_API_KEY" \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "params": {
    "depth": 2,
    "format": "markdown",
    "limit": 420,
    "url": "https://qyuedu.net/chapter-directory/89455"
  }
}'
Response exampleresponse.json
JSONPartial content
{
  "creditUsage": 10,
  "depth": 2,
  "format": "markdown",
  "limit": 420,
  "results": [
    {
      "duration_elasped_ms": 56840,
      "error": "Error getting website url.",
      "metadata": {
        "automation_data": null,
        "crawl_id": null,
        "description": "",
        "domain": "qyuedu.net",
        "title": "",
        "url": null
      },
      "status": 403,
      "url": "https://qyuedu.net/chapter-directory/89455"
    }
  ],
  "url": "https://qyuedu.net/chapter-directory/89455"
}

This example shows part of the response. Copied content is valid JSON.