> ## Documentation Index
> Fetch the complete documentation index at: https://docs.scrapellm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Error handling

> Understanding HTTP status codes, error response formats, retry logic, and troubleshooting strategies for all ScrapeLLM scraper endpoints.

The ScrapeLLM API uses standard HTTP status codes and consistent error response formats across all endpoints.

## HTTP status codes

| Status code | Meaning               | Description                                           |
| ----------- | --------------------- | ----------------------------------------------------- |
| `200`       | Success               | Request completed successfully                        |
| `202`       | Accepted              | Async job submitted successfully                      |
| `400`       | Bad Request           | Request validation failed                             |
| `401`       | Unauthorized          | Missing or invalid API key                            |
| `404`       | Not Found             | Job ID not found or expired                           |
| `408`       | Request Timeout       | AI provider did not respond within the timeout period |
| `429`       | Too Many Requests     | Credit limit reached or rate limit exceeded           |
| `500`       | Internal Server Error | Scraper returned an unexpected error - safe to retry  |
| `503`       | Service Unavailable   | Scraper pool is warming up - retry in a few seconds   |
| `504`       | Gateway Timeout       | Network timeout reaching the scraper                  |

## Error response format

All errors return a JSON body with a `detail` field:

```json theme={null}
{
  "detail": "Human-readable description of the error."
}
```

### Authentication error (401)

```json theme={null}
{
  "detail": "Missing API key. Pass your key in the X-API-Key header or as ?api_key= in the URL."
}
```

### Credit limit (429)

```json theme={null}
{
  "detail": "Credit limit reached. Upgrade your plan or wait for your billing cycle to reset."
}
```

### Timeout (408)

The AI provider did not respond within the `timeout` window. Increase `timeout` (up to 600 seconds) or retry with `bypass_cache=true`.

### Job not found (404)

```json theme={null}
{
  "detail": "Job not found. It may have expired (jobs are retained for 24 hours) or the ID is invalid."
}
```

## Common error types

### Authentication errors (401)

| Cause                                          | Fix                                          |
| ---------------------------------------------- | -------------------------------------------- |
| No `X-API-Key` header and no `?api_key=` param | Add the header or query param                |
| Key is invalid or revoked                      | Regenerate your key in the dashboard         |
| Account has no credits                         | Upgrade plan or wait for billing cycle reset |

### Credit / rate limit (429)

| Cause                              | Fix                                                |
| ---------------------------------- | -------------------------------------------------- |
| Monthly credit allowance exhausted | Upgrade plan or wait for next billing cycle        |
| Scraper queue full                 | Retry after a few seconds with exponential backoff |

### Server errors (500 / 503 / 504)

These are transient errors - it is always safe to retry.

| Code  | Cause                                                |
| ----- | ---------------------------------------------------- |
| `500` | Scraper returned an unexpected error or empty result |
| `503` | Scraper pool is warming up                           |
| `504` | Network timeout to the underlying scraper            |

## Retry logic

Implement exponential backoff for `408`, `429`, `500`, `503`, and `504` errors:

<CodeGroup>
  ```python Python theme={null}
  import time, requests

  def scrape_with_retry(prompt, max_retries=3):
      for attempt in range(max_retries):
          resp = requests.get(
              "https://api.scrapellm.com/scrapers/chatgpt",
              headers={"X-API-Key": "YOUR_API_KEY"},
              params={"prompt": prompt, "country": "US"},
          )
          if resp.status_code == 200:
              return resp.json()
          if resp.status_code in (408, 429, 500, 503, 504) and attempt < max_retries - 1:
              time.sleep(2 ** attempt)  # 1s, 2s, 4s ...
              continue
          resp.raise_for_status()
  ```

  ```javascript JavaScript theme={null}
  async function scrapeWithRetry(prompt, maxRetries = 3) {
    for (let i = 0; i < maxRetries; i++) {
      const resp = await fetch(
        `https://api.scrapellm.com/scrapers/chatgpt?prompt=${encodeURIComponent(prompt)}&country=US`,
        { headers: { "X-API-Key": "YOUR_API_KEY" } }
      );

      if (resp.ok) return resp.json();

      const retryable = [408, 429, 500, 503, 504].includes(resp.status);
      if (retryable && i < maxRetries - 1) {
        await new Promise(r => setTimeout(r, Math.pow(2, i) * 1000));
        continue;
      }
      throw new Error(`HTTP ${resp.status}`);
    }
  }
  ```
</CodeGroup>

<Info>
  Async jobs are automatically retried server-side up to **3 times** before
  being marked `failed`. Credits are restored if all attempts fail.
</Info>

## Async job errors

When an async job reaches `failed` status, the error message is in the `error` field:

```json theme={null}
{
  "job_id": "3f7a2b1c-...",
  "status": "failed",
  "scraper": "chatgpt",
  "error": "execution failed",
  "created_at": "2026-05-11T12:00:00+00:00",
  "completed_at": "2026-05-11T12:01:45+00:00"
}
```

A `failed` job can be resubmitted by making a new request. Credits from the failed job are restored automatically.

## Troubleshooting

### 401 Unauthorized

* Verify the `X-API-Key` header is present and correctly spelled
* Check the key is valid and hasn't been revoked in the dashboard
* Confirm your account has remaining credits

### 429 Too Many Requests

* Check your credit balance in the [dashboard](https://scrapellm.com/dashboard)
* If the queue is full, implement retry with exponential backoff
* Consider using async job mode to avoid holding connections

### 500 / 503 / 504

These are almost always transient. Retry immediately or after a brief delay. If the error persists for more than a few minutes, contact [hello@scrapellm.com](mailto:hello@scrapellm.com) with your `job_id`.

### Amazon Rufus 500

If Rufus returns no products for a very vague query, the endpoint returns `500` with `"Scraper returned an empty result."` - retry with a more specific shopping query. Credits are restored automatically.