Rate limits
All accounts are subject to per-second and per-minute request rate limits. Limits are stored per-user and enforced by the API gateway. When you exceed a limit, you receiveHTTP 429:
429 responses. See Error handling for a code example.
Pattern 1: Async jobs (recommended for batch workloads)
For large batches, submit async jobs and poll for results. This avoids holding open HTTP connections and lets the ScrapeLLM job queue handle concurrency automatically.Pattern 2: Concurrent workers (sync requests)
For smaller batches where you need immediate results, use concurrent sync requests - but respect rate limits.Quick reference
Common questions
Why am I getting 429 errors?
You’ve exceeded your plan’s rate limit. Options:- Implement exponential backoff and retry
- Switch to async job mode - jobs are queued server-side
- Upgrade your plan for higher limits