Bright Data Run Scraper
Run a Bright Data scraper to collect records for URLs or queries. Provide dataset_id from search_scrapers, a method copied exactly from the scraper's collection_methods, and a nonempty input array with one object per URL or query. Check get_scraper_details for that dataset_id and method first, and build each input object from sample_input or input_fields. collect_by_url returns one record per input; discover_by_* methods return up to limit_per_input records per input. The optional limit_per_input applies only to discovery methods, accepts integers from 1 to 1000, and defaults to 10. Every returned record is billed. The call waits up to 45 seconds and returns collected records if complete, or snapshot_id with status "running" if processing continues. Use get_scraper_progress to check the run, then get_scraper_results when ready. Records with an error field explain failed inputs; other record fields depend on the selected scraper and method.
Pricing
| Condition | Charge |
|---|---|
| Each call | 1.8 AE credits per record, based on actual returned records. |
Starting price applies to successful calls with at least 1 returned records. Returned records that fail because of incorrect input are still billed. collect_by_url returns one record per input; for discover_by_*, the discovery limit per input affects the number of records returned, and charges are based on the actual returned records.
Parameters
Optional parameters use the platform defaults when omitted.
| Parameter | Type | Required | Description | Default |
|---|---|---|---|---|
input | array | Yes | List of input objects shaped like sample_input from get_scraper_details, with one object per URL or query to process. | — |
method | string | Yes | One of the scraper collection_methods, exactly as listed (some names are misspelled on purpose) | — |
dataset_id | string | Yes | dataset_id from search_scrapers | — |
limit_per_input | integer | No | discover_by_* only | 10 |
Invocation example
Copy the request and explore its response example.
/agent-api/v1/tool/executeX-Api-KeySubmit first, then query with the returned task_id until the task completes.
curl --request POST \
--url 'https://agentearth.ai/agent-api/v1/tool/execute?tool_name=ae_brightdata_run_scraper' \
--header "X-Api-Key: $AGENTEARTH_API_KEY" \
--header 'Content-Type: application/json' \
--data-raw '{
"params": {
"dataset_id": "gd_le8e811kzy4ggddlq",
"input": [
{
"max_reviews": 25,
"reviews_to_not_include": [],
"url": "https://www.amazon.com/dp/B0B2P9YMFG",
"variation_specific": true
}
],
"method": "collect_by_url"
}
}'{
"description": "task accepted; poll the result via query_tool with this task_id",
"query_tool": "ae_get_result",
"status": "created",
"task_id": "0a4e941f-df47-4e50-8447-4533e13b8bd3"
}This example shows part of the response. Copied content is valid JSON.
Guides using this tool
See how this tool fits into real workflows

