Skip to content
ShipWithIan
Lead Enrichment and Scoring

Module 18 · business · intermediate

Lesson 18 · Lead Enrichment and Scoring

Goal

A lead's email address already tells you a lot: their company's website is usually one request away, and the homepage reveals the industry, the tools they pay for, whether they take online bookings, whether they're hiring. Turn that into a researched, scored lead automatically, without a $500/month data subscription.

Prerequisites

  • Modules 03 (Sheets upsert), 11 (normalizing sources with Code), 14 (rule + AI scoring)

Concepts

  • The website is the dataset. Title, meta description, scripts and links expose industry, tech stack (a wp-content path means WordPress; js.hs-scripts.com means HubSpot), booking tools, hiring and contact details.
  • ICP as data. Industries (keyword lists), target industries and "good tech" live in Settings, so a different offer means editing Settings, not nodes.
  • Explainable scores. Every point comes with a reason ("+10 uses online booking"). Sales teams trust scores they can read.
  • Never let the internet break your workflow. The fetch uses Never Error + Full Response (so a 404 is data, not a crash) and On Error → Continue (so DNS failures and timeouts are data too). AI and Sheets also continue on error. The response always explains what was missing.
  • Mock websites for development. A tiny n8n workflow serves fake homepages. Tests are deterministic and you never hammer real sites while building.
  • Scraping etiquette. Fetch one page per lead, send an honest User-Agent, respect robots.txt and sites' terms, and don't store more personal data than you need.

Build it

Step 1: Demo websites

Import workflow/learner-demo-sites.json (or build it: Webhook GET demo-sites/:site → Code returning HTML per site → Respond to Webhook with Text, response code from the data, header Content-Type: text/html). Publish it. Webhooks with path parameters live under the webhook id: http://localhost:5678/webhook/course-18-demo-sites/demo-sites/acme-dental.

Step 2: Input

  • Webhook Enrich Lead: POST leads/enrich, respond via node.
  • Edit Fields Settings (edit me): offer (for the AI), icp (industries, target_industries, good_tech), hot_threshold 70, warm_threshold 40, sheet_url.
  • Code Normalize Lead (workflow/code/normalize.js): lowercase email; detect free-mail domains; website = given, or https://<business email domain>; hostname via a regex (URL() threw in the Code sandbox here, which silently blanked every website).
  • IF Valid? (email or website) → 400.

Step 3: Fetch the homepage

IF Has Website? → HTTP Request Fetch Website: GET {{ $json.website }}, header User-Agent, Options → Response: Full Response on, Never Error on, format Text; Timeout 10 s; follow up to 5 redirects. Node Settings → On Error → Continue (regular output).

Step 4: Extract signals

Code Extract Signals (extract.js): site_ok only if there's HTML and status < 400; then regexes for title, meta description, h1, emails, phones, social links, a tech map (wordpress: /wp-content|wp-includes/, hubspot: /js\.hs-scripts\.com/, …), booking links, hiring words and the <address>. Visible text (scripts/styles stripped, 4,000 chars) goes to the AI.

Step 5: Score

Code Rule Score (score.js): find the industry by keyword, then add points with reasons, capped at 100.

Step 6: AI profile (optional)

IF Use AI? (not dry run / ai=false, site reachable) → Information Extractor AI Profile with the offer + signals + page text: summary, industry, size_guess, fit_score (0–100 against the offer), pitch_angle. OpenAI gpt-5-mini; On Error → Continue.

Code Combine (combine.js): score = rules, or 50/50 with AI when it succeeded; tier; a clean record with company name (given → site title → email domain), reasons, signals and ai_error if AI failed.

Step 7: Save and alert

IF Dry Run? → respond. Otherwise Google Sheets Save to Sheet: Append or Update Row on sheet Enriched Leads, match on email (columns: email, name, company, domain, industry, score, tier, summary, pitch_angle, signals, enriched_at); On Error → Continue → Code Save Result (saved only if the row came back) → Respond, plus IF Hot Lead? → Alert Sales (Exercise 1).

Test it

./scripts/load-workflow.sh modules/18-lead-enrichment   # loads the demo sites too
./test/test.sh                                          # 21 checks, no credentials, no internet
WITH_CREDS=1 ./test/test.sh                             # + AI profile + sheet row

Limits (real websites)

  • JavaScript-rendered sites (many React/Next.js sites) return little HTML; signals will be thin. Use a rendering service or the AI on whatever text is there.
  • Bot protection (Cloudflare challenges) returns 403/503, which is reported as site_error, not a crash.
  • Detection is heuristic. A /careers link doesn't guarantee open roles. Treat signals as hints and keep the reasons visible.

Common problems

  • Every lead says "no website to check": the website got blanked during normalizing (that's what URL() did here); check Normalize Lead's output.
  • site_ok false for a site you can open: it blocks bots or needs JavaScript; check site_status.
  • Industry always null: your ICP keywords don't match the wording on real sites; add synonyms.
  • Duplicate rows in the sheet: Column to match on must be email.