Tools

ScrapingBee

HTML API web scraping with the api_key injected for you

View as Markdown

Proxies ScrapingBee’s HTML API, injecting the secret api_key query parameter:

GET|POST /scrapingbee   → https://app.scrapingbee.com/api/v1/?api_key=<secret>&<your query>
  • GET scrapes the target URL.
  • POST forwards a POST to the target.

All controls travel in the query string — the target url and any ScrapingBee parameter (render_js, premium_proxy, country_code, extract_rules, …). Your query string is passed through byte-for-byte.

What the gateway injects and strips

ScrapingBee honors three auth channels; the gateway injects one and strips the other two:

  • api_key is injected first, ahead of your parameters, so a partially-encoded target URL can’t swallow it.
  • A client-supplied api_key query param is dropped (in any encoding).
  • The Authorization header is stripped (it’s also a ScrapingBee auth channel, and your gateway token).
  • The X-API-KEY header is dropped — the header spelling of the key.

Examples

$curl "$GATEWAY/scrapingbee?url=https://example.com" \
>  -H "Authorization: Bearer $TOKEN"

The response is ScrapingBee’s, returned unchanged. For the full parameter set, see ScrapingBee’s documentation.