Skip to content
ShipWithIan
AI Voice Agent Backend

Module 13 · ai · intermediate

Lesson 13 · AI Voice Agent Backend

Goal

Build the backend that makes an AI phone agent useful. Voice platforms like Retell AI and Vapi handle speech, turn-taking and the LLM conversation. When the agent needs facts or actions ("is 10 AM free on Tuesday?", "book it"), the platform calls your HTTP endpoint and reads your answer back to the caller. That endpoint is an n8n workflow, and it's the part clients pay for.

Prerequisites

  • Modules 02 (Switch-style routing, items), 04 (reliability), 08 (AI tools concepts)
  • Optional for the end: a Retell AI or Vapi account (both have trial credit)

Concepts

  • Tools = webhooks. In Retell they're custom functions; in Vapi, custom tools. You give the platform a URL, a name, a description and a JSON schema of arguments; the LLM decides when to call it.
  • Normalize at the edge. Retell sends { name, args, call } (or just the args in args-only mode). Vapi sends { message: { type: "tool-calls", toolCallList: [{ id, function: { name, arguments } }] } } and expects { results: [{ toolCallId, result }] } back. One Code node converts any of them into { platform, tool, args, tool_call_id } items, and one formats the reply. Everything in between is platform-independent.
  • Speed. A caller is waiting in silence. Keep tool workflows small (no AI calls inside tools, no slow APIs) and return in well under a second. This workflow answers in ~25 ms.
  • Speakable results. Return a message written for the ear ("9 AM, 10 AM and 2 PM") plus structured fields. The agent can read the message verbatim.
  • Static data. $getWorkflowStaticData('global') is a small JSON store attached to the workflow. It persists across production executions (published workflow), not manual test runs. Perfect for a credential-free booking demo; swap for Google Calendar or a database in production.
  • Post-call webhook. Platforms POST the transcript when the call ends. That's where slow AI work belongs: summaries, intent, follow-ups.

Build it

Step 1: Endpoint and settings

  • Webhook Voice Tools API: POST, path voice/tools, respond with a Respond to Webhook node.
  • Edit Fields Business Settings (Include Other Input Fields on): business_name, open_hour = 9, close_hour = 17, open_days = {{ [1,2,3,4,5] }} (Luxon weekday numbers, Mon=1), default_service.

Step 2: Normalize

Code Normalize Tool Calls (see workflow/code/normalize.js): loop over input items. If body.message.toolCallList exists → Vapi, one item per call (parse arguments if it's a string). Else if body.name exists → Retell. Else → args-only/plain, with the tool name from ?tool=.

Step 3: Route

Switch Route Tool on {{ $json.tool }} equals check_availability / book_appointment / cancel_appointment / lookup_customer, renamed outputs, fallback output → Edit Fields Unknown Tool with a polite result.

Step 4: The tools (Code nodes named after the tool)

All four read $('Business Settings').first().json and $getWorkflowStaticData('global').bookings, and return { ...json, result: { ok, message, … } }:

  • check_availability (availability.js): validate the date (real, not past, open day), build hourly slots from opening hours, remove booked ones, and produce a sentence like "On Monday, November 2 we have 9 AM, 11 AM and 2 PM available."
  • book_appointment (book.js): check for missing fields (ask for exactly what's missing), opening hours and conflicts; create BK-1234; confirm with the id spelled out ("B K 1 2 3 4") so text-to-speech reads it clearly.
  • cancel_appointment (cancel.js): by booking id, or phone + date.
  • lookup_customer (lookup.js): demo customer list + upcoming bookings for that phone.

Step 5: Reply in the platform's format

Code Format for Platform (format.js): for Vapi, return { results: [{ toolCallId, result: JSON.stringify(result) }] }; for Retell/plain, return the result object. Then Respond to Webhook with {{ $json }}. Publish the workflow, because static data only persists for production runs.

Step 6: Post-call summary

  • Webhook Call Ended (POST voice/call-ended, respond immediately).
  • Code Normalize Call (normalize-call.js): accept Retell call_ended and Vapi end-of-call-report; return nothing for other events.
  • Information Extractor Summarize Call + OpenAI gpt-5-mini: summary, caller_intent, follow_up_needed, follow_up_reason.
  • Edit Fields Call Record → IF Needs Follow-up? → placeholders for alerts and logging (Exercises 2–3).

Step 7: Connect a real voice agent (optional)

Your n8n must be reachable from the internet: use your production server (Module 12) or a tunnel such as ngrok http 5678.

  • Retell: in your agent, add a Custom Function per tool, URL https://<your-host>/webhook/voice/tools, method POST, with a description and parameters, e.g. check_availability: { "date": { "type": "string", "description": "YYYY-MM-DD" } }. Set the agent's webhook URL to /webhook/voice/call-ended.
  • Vapi: create custom tools (type function) with the same names and parameters and server URL /webhook/voice/tools; set the assistant's server URL for end-of-call reports to /webhook/voice/call-ended.
  • Prompt the agent with today's date and timezone, the rule "always check availability before booking", and "read tool messages naturally".
  • Log one real request first (open the execution) and confirm the payload matches what Normalize Tool Calls expects; platforms do change their formats.

Test it

./test/test.sh                 # 16 checks, no credentials
WITH_CREDS=1 ./test/test.sh    # + sends a sample transcript for the AI summary

Common problems

  • Bookings disappear between calls: you're running the workflow manually from the editor. Static data persists only for published, production executions.
  • Vapi says the tool returned nothing: the reply must be { results: [{ toolCallId, result }] } with the exact toolCallId from the request.
  • The agent says weird times: return speakable text ("2 PM"), not "14:00".
  • Long silences on calls: something slow is inside a tool (an AI call, an external API). Move it to the post-call flow, or make the agent say "one moment" before calling the tool.
  • Two tool calls in one Vapi request: only one result comes back (verified), because the items take different Switch branches. Fix it in Exercise 1.