batch_scrape - Charter Docs

POST

firecrawl_batch_scrape.py

import asyncio

from charter.packs import firecrawl

result = asyncio.run(firecrawl.batch_scrape.ainvoke({"urls": ...}))

With firecrawl.RESTRICTED_KEY, requires batch-scrape.

Body

Field	Type	Description
`formats`	FormatType \| Format[]	Output formats to include in the response. Strings or objects. The server applies markdown when this is absent.
`onlyMainContent`	`boolean`	Only return the main content of the page excluding headers, navs, footers, etc. The server applies true when this is absent.
`onlyCleanContent`	`boolean`	Beta. LLM pass over markdown to remove residual boilerplate that onlyMainContent can miss.
`includeTags`	`string`[]	Tags to include in the output.
`excludeTags`	`string`[]	Tags to exclude from the output.
`maxAge`	`integer`	Returns a cached version of the page if it is younger than this age in milliseconds. The server applies 172800000 (2 days) when this is absent.
`minAge`	`integer`	When set, the request only checks the cache and never triggers a fresh scrape.
`headers`	map of `string`	Headers to send with the request.
`waitFor`	`integer`	Specify a delay in milliseconds before fetching the content. The server applies 0 when this is absent.
`mobile`	`boolean`	Emulate scraping from a mobile device.
`skipTlsVerification`	`boolean`	Skip TLS certificate verification when making requests.
`timeout`	`integer`	Timeout in milliseconds. The server applies 60000 when this is absent. ≥ 1000; ≤ 300000
`parsers`	`string` \| PdfParser[]	Controls how files are processed during scraping.
`actions`	Action[]	Actions to perform on the page before grabbing the content.
`location`	ScrapeLocation	Location settings for the request.
`removeBase64Images`	`boolean`	Removes all base64 images from the markdown output.
`blockAds`	`boolean`	Enables ad-blocking and cookie popup blocking.
`proxy`	`"basic" \| "enhanced" \| "auto"`	Specifies the type of proxy to use.
`storeInCache`	`boolean`	If true, the page will be stored in the Firecrawl index and cache.
`lockdown`	`boolean`	Serve from cache only and never make an outbound request. On miss, returns 404 SCRAPE_LOCKDOWN_CACHE_MISS.
`redactPII`	`boolean` \| RedactPIIOptions	Redact personally identifiable information from returned markdown. Pass true for defaults, or an object to tune it.
`profile`	BrowserProfile	Persistent browser storage across scrape and interact sessions.
`threatProtection`	ThreatProtectionOverride	Per-request threat protection override. Enterprise feature.
`auditMetadata`	AuditMetadata	User attribution included with SIEM logging events when SIEM is enabled.
`urls` *	`string`[]	The URLs to scrape
`webhook`	Webhook	Webhook specification for batch scrape lifecycle events.
`maxConcurrency`	`integer`	Maximum number of concurrent scrapes for this batch.
`ignoreInvalidURLs`	`boolean`	If invalid URLs are specified, they are ignored and returned in invalidURLs instead of failing the request.
`zeroDataRetention`	`boolean`	If true, this will enable zero data retention for this batch scrape. To enable this feature, please contact help@firecrawl.dev

​Body

Body