> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.auxiliar.ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.auxiliar.ai/_mcp/server.

# Crawlbase

Proxies [Crawlbase](https://crawlbase.com)'s Crawling API, injecting the secret `token` query parameter:

```
GET|POST /crawlbase      → https://api.crawlbase.com/?token=<normal>&<your query>
GET|POST /crawlbase/js   → https://api.crawlbase.com/?token=<javascript>&<your query>
```

Crawlbase issues **two** tokens, and the **route picks which one is injected**:

| Route           | Token      | Behavior                                  |
| --------------- | ---------- | ----------------------------------------- |
| `/crawlbase`    | Normal     | Fetches static HTML                       |
| `/crawlbase/js` | JavaScript | Loads the page in a real headless browser |

All controls travel in the **query string** — the target `url` and any Crawlbase parameter (`format`, `country`, `page_wait`, …). Your query string is passed through byte-for-byte.

## What the gateway injects and strips

* **`token` is injected first**, ahead of your parameters, so a partially-encoded target URL can't swallow it. The route (`/crawlbase` vs `/crawlbase/js`) selects which token.
* **A client-supplied `token` query param is dropped** (in any encoding), so the injected token can't be overridden.
* **The `Authorization` header is stripped** before forwarding — Crawlbase has no use for your gateway token.

## Examples

```bash
curl "$GATEWAY/crawlbase?url=https://example.com" \
  -H "Authorization: Bearer $TOKEN"
```

```bash
curl "$GATEWAY/crawlbase/js?url=https://example.com" \
  -H "Authorization: Bearer $TOKEN"
```

The response is Crawlbase's, returned unchanged. For the full parameter set, see [Crawlbase's documentation](https://crawlbase.com/docs/crawling-api).