diff --git a/workflows/Scrape & Analyze Google Ads with Bright Data API and AI for Email Reports-6256/readme-6256.md b/workflows/Scrape & Analyze Google Ads with Bright Data API and AI for Email Reports-6256/readme-6256.md new file mode 100644 index 000000000..55f722ad4 --- /dev/null +++ b/workflows/Scrape & Analyze Google Ads with Bright Data API and AI for Email Reports-6256/readme-6256.md @@ -0,0 +1,394 @@ +Scrape & Analyze Google Ads with Bright Data API and AI for Email Reports + +https://n8nworkflows.xyz/workflows/scrape---analyze-google-ads-with-bright-data-api-and-ai-for-email-reports-6256 + + +# Scrape & Analyze Google Ads with Bright Data API and AI for Email Reports + +### 1. Workflow Overview + +This workflow automates the process of scraping Google Ads data via the Bright Data API, analyzing the ad content using AI models, and generating structured email reports summarizing key insights. Its primary use case is for marketing analysts or strategists who want to track keyword-triggered Google Ads, understand advertiser messaging strategies, and receive summarized performance reports via email. + +The workflow is logically divided into the following blocks: + +- **1.1 Input Reception & Keyword Retrieval:** Trigger and fetch list of keywords from Google Sheets. +- **1.2 Google Ads Scraping:** For each keyword and country code, scrape Google Search results JSON including ads via Bright Data API. +- **1.3 Ads Filtering & Extraction:** Filter results to ensure ads exist, then extract top and bottom ads arrays. +- **1.4 AI Analysis of Ads:** Use multiple AI language models to analyze ads for value propositions, messaging style, and extensions. +- **1.5 Site Link Extensions Analysis:** Extract and categorize site link extensions from ads. +- **1.6 Keyword-to-Ad Mapping:** Map keywords to ads and analyze advertiser reuse and messaging differences. +- **1.7 Report Generation & Emailing:** Combine AI insights, generate an HTML report, and send it via Gmail. + +--- + +### 2. Block-by-Block Analysis + +#### 2.1 Input Reception & Keyword Retrieval + +- **Overview:** +Starts the workflow on manual trigger and retrieves a list of keywords and country codes from a Google Sheet to be processed. + +- **Nodes Involved:** + - When clicking ‘Execute workflow’ (Manual Trigger) + - Get Keywords (Google Sheets) + - Filter1 (Filter) + +- **Node Details:** + - **When clicking ‘Execute workflow’** + - Type: Manual Trigger + - Role: Entry point to start the workflow manually. + - Connections: Outputs to Get Keywords. + - Edge cases: No major failure expected. + - **Get Keywords** + - Type: Google Sheets + - Role: Reads keywords and country codes from a specified Google Sheet tab. + - Config: Document ID and sheet GID set to a shared public Google Sheet. + - Credentials: Uses Google Sheets OAuth2 credentials. + - Outputs: Rows with columns including `Keyword` and `country code`. + - Edge cases: OAuth token expiration, empty sheet, or connectivity issues. + - **Filter1** + - Type: Filter + - Role: Filters out any records with empty keyword field. + - Config: Condition to pass only if `Keyword` is not empty. + - Connections: Passes filtered keywords to Loop Over Items. + - Edge cases: No keywords pass filter resulting in empty processing. + +--- + +#### 2.2 Google Ads Scraping + +- **Overview:** +Loops over each keyword/country code, submits scraping requests to Bright Data API to fetch Google Search JSON results including ads. + +- **Nodes Involved:** + - Loop Over Items (Splitting batch to single items) + - set keyword (Set) + - Fetch Google Search Results JSON (HTTP Request) + - filter ads (Filter) + - ads found (Set) + - No Operation, do nothing1 (NoOp) + +- **Node Details:** + - **Loop Over Items** + - Type: SplitInBatches + - Role: Processes one keyword item at a time to handle rate limits and sequential scraping. + - Edge cases: Batch size or rate limits causing delays. + - **set keyword** + - Type: Set + - Role: Formats the current item to create `search_term` and `country code` variables used in scraping request. + - Key Expressions: Sets `search_term` = Keyword, `country code` as is. + - **Fetch Google Search Results JSON** + - Type: HTTP Request + - Role: Calls Bright Data API with parameters for zone (serp_api1), Google search URL with query, country, and format. + - Config: POST request, async param true, uses HTTP header authentication with Bright Data credentials. + - Expressions: URL dynamically built with `search_term` and `country code`. + - Edge cases: API errors, rate limits, malformed requests or auth errors. Has error continuation configured. + - **filter ads** + - Type: Filter + - Role: Checks if `top_ads` array from the JSON response has at least one element to proceed. + - Edge cases: No ads found causing filter to skip. + - **ads found** + - Type: Set + - Role: Extracts `top_ads` and `bottom_ads` arrays from the response JSON for downstream processing. + - **No Operation, do nothing1** + - Type: NoOp + - Role: Acts as a placeholder for the error branch of the HTTP request. + +--- + +#### 2.3 Ads Filtering & Extraction + +- **Overview:** +Extracts individual ads from the combined top and bottom ads arrays for detailed analysis and writes raw ad data to Google Sheets. + +- **Nodes Involved:** + - extract ads (Code) + - extensions (Aggregate) + - value proposition1 (Aggregate) + - keywords (Aggregate) + - Split ads (SplitOut) + - Edit Fields (Set) + - Top ads (Google Sheets) + +- **Node Details:** + - **extract ads** + - Type: Code + - Role: Combines `top_ads` and `bottom_ads` arrays and outputs each ad as separate workflow items for parallel processing. + - Code: Maps combined ads array to individual JSON items. + - **extensions, value proposition1, keywords** + - Type: Aggregate + - Role: Aggregates all ad items' data except excluded fields to prepare batches for AI analysis steps. + - **Split ads** + - Type: SplitOut + - Role: Splits combined ad arrays field into individual items for Google Sheets insertion. + - **Edit Fields** + - Type: Set + - Role: Adds a static `keyword` field ("Business automations") for context in the Google Sheet. + - **Top ads** + - Type: Google Sheets + - Role: Appends cleaned and formatted top ads data to a specific Google Sheet tab for records. + - Credentials: Google Sheets OAuth2 account. + - Edge cases: Sheet access or quota limitations. + +--- + +#### 2.4 AI Analysis of Ads + +- **Overview:** +Runs multiple AI language models to analyze ad messaging, value propositions, and extract structured insights. + +- **Nodes Involved:** + - Value Proposition & Messaging Analysis (Chain LLM) + - Site Link Extension Mining (Chain LLM) + - Keyword-Ad Mapping (Chain LLM) + - OpenRouter Chat Model (LM Chat) + - Google Gemini Chat Model (LM Chat) + - Structured Output Parser nodes (Output Parsers) + - value proposition (Set) + - site links (Set) + - keywords1 (Set) + +- **Node Details:** + - **Value Proposition & Messaging Analysis** + - Type: Chain LLM + - Role: Analyzes ads' JSON data to identify core value propositions, tone, marketing devices, and common phrases. + - Prompt: Detailed instructions to classify emotional hooks, rational benefits, guarantees, and marketing claims. + - Output: Structured JSON with advertiser insights. + - **Site Link Extension Mining** + - Type: Chain LLM + - Role: Extracts and categorizes site link extensions and CTA strategies from ad extensions JSON. + - Output: Structured JSON listing site link text, URLs, types, and UTM presence. + - **Keyword-Ad Mapping** + - Type: Chain LLM + - Role: Maps keywords to ads, identifies advertiser reuse, clusters related keywords, and compares messaging differences. + - Output: Structured JSON summarizing keyword-ad relations and competitive insights. + - **OpenRouter Chat Model & Google Gemini Chat Model** + - Type: Language Model Chat nodes + - Role: Serve as AI engines for the chain LLM nodes, providing alternative AI model endpoints (OpenRouter, Google Gemini). + - Credentials: Requires respective API credentials. + - **Structured Output Parsers** + - Type: Output Parser + - Role: Parses AI output to ensure it conforms to the expected JSON schema. + - **value proposition, site links, keywords1** + - Type: Set + - Role: Stores AI output from each analysis step into named JSON arrays for merging later. + +--- + +#### 2.5 Report Generation & Emailing + +- **Overview:** +Merges all AI insights and generates a detailed, structured HTML email report sent to a specified recipient. + +- **Nodes Involved:** + - Merge1 (Merge) + - Generate HTML (Agent LLM) + - Send a message (Gmail) + +- **Node Details:** + - **Merge1** + - Type: Merge + - Role: Combines AI insights for value proposition, site links, and keyword-ad mapping by position to produce a unified JSON input for report generation. + - **Generate HTML** + - Type: Agent LLM + - Role: Uses the merged JSON data to generate a professional HTML-formatted email body summarizing all insights. + - Prompt: Specifies formatting with HTML tags `

`, `