Bright Data Block

The Bright Data block brings web intelligence into your workflows. Use it to scrape any URL (bypassing anti-bot protections), run search engine queries across Google, Bing, or DuckDuckGo, and discover intent-ranked web pages — all through Bright Data's network.

Overview

When to Use

• Scrape the raw HTML or structured JSON of a specific web page, bypassing CAPTCHAs and IP blocks automatically.
• Run a search engine query and collect ranked SERP results programmatically across Google, Bing, or DuckDuckGo.
• Discover URLs related to a starting page using AI-powered intent-based ranking (up to 1,000 results).
• Gather competitive or market data (e.g., product listings, pricing pages) at scale.
• Feed scraped page content into downstream agents or knowledge bases for RAG pipelines.
• Pull geo-targeted search results for a given query by specifying a country code.

Configuration

Operation

Required. Select which Bright Data capability to invoke. Options:

The choice here determines which other fields appear.

Zone

Required. Your Bright Data zone identifier from the Bright Data dashboard (e.g., web_unlocker1 for scraping, serp_api1 for SERP). This field is visible for all operations.

URL

The target page URL to scrape (e.g., https://example.com/page). Shown when operation is Scrape URL or Discover URLs.

Search Query

The search query string to run (e.g., best laptops 2024). Shown when operation is SERP Search.

Format

Output format for scraped pages. Shown when operation is Scrape URL. Options:

Defaults to raw when not specified.

Search Engine

Which engine to query. Shown when operation is SERP Search. Options:

Defaults to google when not specified.

Inputs & Outputs

• Inputs:

  • operation (string) — Operation to perform (brightdata_scrape_url, brightdata_serp_search, or brightdata_discover)
  • zone (string) — Bright Data zone name from your dashboard
  • url (string) — Target URL; used by Scrape URL and Discover URLs operations
  • query (string) — Search query; used by SERP Search operation
  • format (string) — Output format (raw or json); used by Scrape URL operation
  • searchEngine (string) — Search engine to use (google, bing, or duckduckgo); used by SERP Search operation

• Outputs (block-level):

  • html (string) — Raw HTML content (from Scrape URL with format: raw)
  • json (json) — Structured data (from Scrape URL with format: json)
  • results (json) — Search or discovery results (from SERP Search or Discover URLs)

Tools

Bright Data Scrape URL (brightdata_scrape_url) — Fetches the content of any URL via the Bright Data Web Unlocker. Automatically bypasses anti-bot protections, CAPTCHAs, and IP blocks. Requires apiKey (your Bright Data API token) and zone. Returns content (raw HTML or JSON string), url (echoed URL), and statusCode (HTTP response code).

Bright Data SERP Search (brightdata_serp_search) — Submits a search query to Google, Bing, DuckDuckGo, or Yandex via the Bright Data SERP API and returns structured results. Requires apiKey, zone, and query. Optional params include searchEngine, country, language, and numResults (up to the engine's limit). Returns results (array of {title, url, description, rank}), query, and searchEngine.

Bright Data Discover (brightdata_discover) — AI-powered web discovery that finds and ranks web pages by intent. Submits a job and polls until completion (up to 5 minutes). Returns up to 1,000 results with optional cleaned page content for RAG and verification. Requires apiKey and query. Optional params include numResults, intent, includeContent, format, language, and country. Returns results (array of {url, title, description, relevanceScore, content}), query, and totalResults.

YAML Example

brightdata_1:
  type: brightdata
  name: "Bright Data"
  inputs:
    operation: "brightdata_serp_search"
    zone: "{{BRIGHTDATA_SERP_ZONE}}"
    query: "{{search_input.result}}"
    searchEngine: "google"
  connections:
    outgoing:
      - target: next-block-id