Skip to content
ShipWithIan
AI Job Scout

Module 11 · ai · intermediate

Lesson 11 · AI Job Scout

Goal

Combine everything so far: multiple data sources, a Code node for messy normalization, cheap filtering before expensive AI calls, and per-item AI scoring. It's also a tool you'll use yourself to find automation work.

Prerequisites

  • Modules 02 (split/sort/limit/aggregate), 06 (Information Extractor)

Concepts

  • Normalize early: every source has its own shape (RemoteOK position, WWR title = "Company: Title"). Map them all to one shape right after fetching, and everything downstream stays simple.
  • Code node: when mapping needs real logic (filtering a legal-notice item, splitting strings, stripping HTML), JavaScript is clearer than five Edit Fields nodes. Return [{ json: {...} }, …].
  • Merge (Append): waits for both inputs and outputs all items from both.
  • Filter before AI: keywords → newest → top N, and then AI. Every item that reaches Score Job costs a request, so limit caps it (max 25).
  • Whole-word matching: includes('llm') matches "enrollment". A small regex with word boundaries fixes it.
  • Per-item AI: the Information Extractor runs once per job; $('Top N').item.json pairs each result back with its job.

Build it

Step 1: Triggers and settings

  • Schedule Trigger Every 4 Hours and Webhook Scout API (GET jobs/scout, respond via node) → Edit Fields Settings (edit profile):
    • keywords (Array) = {{ ($json.query?.keywords ?? 'n8n,zapier,make.com,automation engineer,…').split(',').map(k => k.trim().toLowerCase()).filter(Boolean) }}
    • min_score = {{ Number($json.query?.min_score ?? 60) }}, max_jobs = {{ Math.min(Number($json.query?.limit ?? 10), 25) }}, dry_run = {{ $json.query?.dry_run === 'true' }}
    • profile: your skills, location/timezone and preferences, written as a paragraph

Step 2: Fetch both sources (in parallel)

Connect Settings to both:

  • HTTP Request Fetch RemoteOK: GET https://remoteok.com/api with header User-Agent: Mozilla/5.0 (n8n job scout). The API returns an array, and n8n splits it into items automatically.
  • RSS Read Fetch We Work Remotely: https://weworkremotely.com/categories/remote-programming-jobs.rss
  • On both, node Settings → Retry On Fail (3 tries, 2000 ms) and On Error → Continue. Job boards drop connections now and then, and one flaky source shouldn't kill the run. The Normalize nodes skip the error item.

Step 3: Normalize (Code nodes)

Normalize RemoteOK: drop items without position (the first item is a legal notice), strip HTML from description, map to {source, title, company, url, tags, location, salary, posted_at, description}.

Normalize WWR: split title on ": " into company + title, url = link, posted_at = isoDate, description = contentSnippet.

See workflow/learner.json for the full code.

Step 4: Merge, filter, sort, limit

  1. Merge All Jobs (Append), RemoteOK into input 1, WWR into input 2.
  2. Filter Matches Keywords: Boolean is true on {{ $('Settings (edit profile)').first().json.keywords.some(k => new RegExp('(^|[^a-z0-9])' + k.replace(/[.*+?^${}()|[\]\\]/g, '\\$&') + 's?($|[^a-z0-9])').test([$json.title, $json.tags.join(' '), $json.description].join(' ').toLowerCase())) }}
  3. Sort Newest First by posted_at descending → Limit Top N = max_jobs.

Step 5: Dry run or score

IF Dry Run? (dry_run is true):

  • true → Aggregate Collect Matches → Respond with count and slim jobs
  • false → Information Extractor Score Job with your profile + the job as text, attributes score (number, 0–100 with a rubric), fit_reasons, red_flags, pitch_angle; system prompt "blunt, practical career advisor… never invent facts". Model: OpenAI gpt-5-mini.

Step 6: Report

Edit Fields Scored Job (job fields from $('Top N').item.json + clamped score) → Filter Good Fit Only (score ≥ min_score, Always Output Data on) → Aggregate → Edit Fields Build Report (jobs sorted by score, count, message text) → IF Called via API? → Respond / Send Alert (Exercise 1).

Test it

./test/test.sh                 # dry run against live boards
WITH_CREDS=1 ./test/test.sh    # + AI scoring of 5 jobs
curl 'http://localhost:5678/webhook/jobs/scout?limit=8' | jq -r '.message'

Common problems

  • RemoteOK returns 403 or nothing: the User-Agent header is missing.
  • Merge outputs nothing: one source failed or returned zero items. Check each fetch node.
  • Irrelevant matches: generic keywords ("automation", "workflow") appear in many descriptions. Use specific phrases.
  • Slow / costly runs: lower limit. Every job reaching Score Job is one AI request.
  • Same jobs every 4 hours: add "seen jobs" memory (Exercise 2).