> ## Documentation Index
> Fetch the complete documentation index at: https://docs.scrapellm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Work with AI

> How to use ScrapeLLM within AI pipelines, agents, and LLM workflows - including copy-paste context for AI assistants.

ScrapeLLM is designed to work inside AI pipelines. This page covers how to provide ScrapeLLM context to AI assistants and how to wire it into agents and LLM workflows.

## Documentation index for AI tools

Paste the following URL into any AI assistant (Claude, ChatGPT, Cursor, etc.) to give it full context about the ScrapeLLM API:

```
https://docs.scrapellm.com/llms.txt
```

This file lists every documentation page and OpenAPI spec in a format optimised for LLM consumption.

## Copy-paste context

If you want to give an AI assistant a quick summary of ScrapeLLM without fetching the docs, copy the block below:

```
ScrapeLLM is a REST API that extracts real AI responses from ChatGPT, Perplexity, 
Grok, Copilot, Gemini, Google AI Mode, and Amazon Rufus as structured JSON.

Authentication: pass your API key in the X-API-Key header.
Base URL: https://api.scrapellm.com

Sync request (GET):
  GET /scrapers/{scraper}?prompt=...&country=US
  Headers: X-API-Key: YOUR_KEY
  Returns: { scraper, status, result, links/sources/citations, credits_used, ... }

Async request (POST → poll):
  POST /scrapers/{scraper}/jobs?prompt=...&country=US
  Returns: { job_id, status: "pending" }
  Poll: GET /jobs/{job_id} until status is "done" or "failed"

Scrapers: chatgpt, perplexity, grok, copilot, gemini, google_ai_mode, amazon_rufus
Credits: 3 per request. Free tier: 1,500/month.
```

## Using ScrapeLLM as an AI agent tool

ScrapeLLM works naturally as a tool in any LLM agent framework. Here are minimal tool definitions:

<CodeGroup>
  ```python LangChain theme={null}
  from langchain.tools import tool
  import requests

  @tool
  def scrape_chatgpt(prompt: str, country: str = "US") -> dict:
      """Query ChatGPT and return the real web UI response as structured JSON,
      including cited sources and internal search queries."""
      return requests.get(
          "https://api.scrapellm.com/scrapers/chatgpt",
          headers={"X-API-Key": "YOUR_API_KEY"},
          params={"prompt": prompt, "country": country},
      ).json()

  @tool
  def scrape_perplexity(prompt: str, country: str = "US") -> dict:
      """Query Perplexity AI and return the response with numbered citations and source snippets."""
      return requests.get(
          "https://api.scrapellm.com/scrapers/perplexity",
          headers={"X-API-Key": "YOUR_API_KEY"},
          params={"prompt": prompt, "country": country},
      ).json()
  ```

  ```python OpenAI function calling theme={null}
  import openai, requests

  tools = [
      {
          "type": "function",
          "function": {
              "name": "scrape_ai",
              "description": "Query an AI provider (chatgpt, perplexity, grok, gemini, copilot, google_ai_mode, amazon_rufus) and return the real UI response as JSON.",
              "parameters": {
                  "type": "object",
                  "properties": {
                      "scraper": {
                          "type": "string",
                          "enum": ["chatgpt", "perplexity", "grok", "copilot", "gemini", "google_ai_mode", "amazon_rufus"],
                      },
                      "prompt": {"type": "string"},
                      "country": {"type": "string", "default": "US"},
                  },
                  "required": ["scraper", "prompt"],
              },
          },
      }
  ]

  def scrape_ai(scraper: str, prompt: str, country: str = "US") -> dict:
      return requests.get(
          f"https://api.scrapellm.com/scrapers/{scraper}",
          headers={"X-API-Key": "YOUR_API_KEY"},
          params={"prompt": prompt, "country": country},
      ).json()
  ```

  ```typescript Vercel AI SDK theme={null}
  import { tool } from "ai";
  import { z } from "zod";

  const scrapeAI = tool({
    description:
      "Query an AI provider and return the real UI response as structured JSON, including sources and citations.",
    parameters: z.object({
      scraper: z.enum([
        "chatgpt", "perplexity", "grok", "copilot",
        "gemini", "google_ai_mode", "amazon_rufus",
      ]),
      prompt: z.string(),
      country: z.string().default("US"),
    }),
    execute: async ({ scraper, prompt, country }) => {
      const params = new URLSearchParams({ prompt, country });
      const res = await fetch(
        `https://api.scrapellm.com/scrapers/${scraper}?${params}`,
        { headers: { "X-API-Key": process.env.SCRAPELLM_API_KEY! } }
      );
      return res.json();
    },
  });
  ```
</CodeGroup>

## Common AI pipeline patterns

### Cross-provider brand monitoring

Query the same prompt across multiple AI providers in parallel to measure your brand's AI visibility:

```python theme={null}
import asyncio, aiohttp

async def monitor_brand(prompt: str, scrapers: list[str], country: str = "US"):
    async with aiohttp.ClientSession() as session:
        tasks = [
            session.get(
                f"https://api.scrapellm.com/scrapers/{s}",
                headers={"X-API-Key": "YOUR_API_KEY"},
                params={"prompt": prompt, "country": country},
            )
            for s in scrapers
        ]
        responses = await asyncio.gather(*tasks)
        return {
            s: await r.json()
            for s, r in zip(scrapers, responses)
        }

results = asyncio.run(monitor_brand(
    "What CRM do sales teams recommend?",
    scrapers=["chatgpt", "perplexity", "grok", "gemini"],
))

for scraper, data in results.items():
    print(f"{scraper}: {data['result'][:200]}")
```

### Feed AI responses into another LLM

Pipe ScrapeLLM output directly into an LLM for synthesis or analysis:

```python theme={null}
import requests
from openai import OpenAI

client = OpenAI()

# 1. Get the real AI response
scrape = requests.get(
    "https://api.scrapellm.com/scrapers/perplexity",
    headers={"X-API-Key": "YOUR_API_KEY"},
    params={"prompt": "Best project management tools for remote teams?"},
).json()

# 2. Feed it to GPT-4o for analysis
analysis = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You analyse AI search results for brand mentions."},
        {"role": "user", "content": f"Analyse this Perplexity response:\n\n{scrape['result']}\n\nSources: {scrape['sources']}"},
    ],
)

print(analysis.choices[0].message.content)
```