Skip to content
ShipWithIan
Document Intake and Routing

Module 17 · business · intermediate

Lesson 17 · Document Intake and Routing

Goal

Offices drown in PDFs: invoices, contracts, statements, receipts, each downloaded, renamed and dragged into a folder by hand. Build an intake that does it automatically and consistently, and that knows when to ask a human.

Prerequisites

  • Module 07 (PDF upload + Extract From File + Information Extractor)
  • For the live part: a Google Drive credential and, optionally, an OpenAI key

Concepts

  • Rules first, AI second, human last. Keyword scoring handles clear documents instantly and for free; AI handles the rest; low confidence goes to a review folder. Each layer is cheaper than the next.
  • Confidence. The rule classifier's confidence combines how far the top type leads the runner-up with how much evidence it found (top/(top+second) × min(1, top/3)). The AI returns its own. Below review_threshold (0.6) → human review.
  • Know who "we" are. Documents mention both sides. our_names in Settings lets the rules pick the other party as counterparty (the provider in a contract, not your own firm).
  • Graceful degradation. AI Classify and the Drive upload both run with On Error → Continue. No OpenAI key → rules decide and the response includes ai_error; Drive down → uploaded: false with the reason. The office still gets an answer.
  • Binary data across nodes. Code and AI nodes don't carry the uploaded file along. Attach File reads it back from the webhook node ($('New Document').first().binary) and renames it before upload.
  • A naming convention is the product. date - Type - Counterparty - Reference.pdf makes folders sortable and searchable. That consistency is what clients notice.

Build it

Step 1: Upload and settings

  • Webhook New Document: POST docs/intake, respond via node.
  • Edit Fields Settings (edit me): review_threshold 0.6, folders (Drive folder id per type + needs_review), our_names (your company names).
  • IF Has PDF? → 400 if no PDF (same check as Module 07).
  • Extract From File Extract PDF Text: PDF, binary field from the upload, Join Pages, Max Pages 5.

Step 2: Rule classifier

Code Rule Classifier (workflow/code/rules.js):

  • Classify: count keyword hits per type (e.g. contract: agreement, by and between, governing law, termination…); top type wins if it has ≥ 2 hits, otherwise other.
  • Date: statement period end → labeled date (Invoice Date, Date, as of, dated) → first date; supports 2026-09-15, October 1, 2026, 09/15/2026.
  • Amount: closing balance → last total that isn't a subtotal (regex lookbehind (?<!sub)total) → largest $ amount.
  • Reference: Invoice Number, Agreement No., Receipt #, PO Number, Account Number.
  • Counterparty: for contracts, the party in "by and between A (…) and B (…)" that isn't us; else the letterhead line. ALL-CAPS names become Title Case (keeping LLC/LLP/Inc.).

Try it: curl -F "file=@assets/samples/invoice.pdf;type=application/pdf" 'http://localhost:5678/webhook-test/docs/intake?dry_run=true'.

Step 3: AI classification (optional)

IF Use AI? (not dry run, not ai=false, has text) → Information Extractor AI Classify (attributes: doc_type from the fixed list, confidence 0–1, counterparty "not our own firm", document_date YYYY-MM-DD, reference, amount, summary) + OpenAI gpt-5-mini. Settings → On Error → Continue (regular output).

Step 4: Decide

Code Decide Route (decide.js): use AI fields when AI succeeded, else rules (and record ai_error); needs_review if no text, type other, or confidence below threshold; build the filename (cleaned of / \ : * ? " < > |) and pick folder_id.

Step 5: File it

IF Dry Run? → Respond with the decision. Otherwise:

  • Code Attach File (attach.js): copy the uploaded binary to data with the new fileName.
  • Google Drive Upload to Drive: Resource File, Upload, Input Data Field data, name {{ $json.filename }}, parent folder By ID {{ $json.folder_id }}. On Error → Continue.
  • Code Upload Result (result.js): uploaded only if Drive returned an id.
  • Respond, plus IF Needs a Human? (review or failed upload) → Notify Office (Exercise 1).

Test it

./test/test.sh                  # 19 checks, no credentials
WITH_CREDS=1 ./test/test.sh     # AI classification + real Drive upload

Common problems

  • Everything goes to review: the PDF has no text layer (scanned). Check text_length in Rule Classifier; add OCR (Exercise 3).
  • Counterparty is your own company: add every variant of your name to our_names.
  • Amount is the subtotal: the document labels the final total differently ("Amount payable"); add that phrase to the total regex.
  • Upload says "no file id returned": the Drive credential or a folder id in Settings is missing or wrong.
  • File uploaded without content: Attach File must take the binary from the webhook node; Extract From File doesn't pass it along by default.