n8n workflows and DSPy — programmatic prompting — Agentic AI Engineering

Why this matters

Two complementary skills that round out a senior agentic AI engineer:

n8n — the open-source, self-hostable workflow automation platform. In 2026 it is the de-facto standard "low-code AI plumbing" — used to wire agents, MCP servers, Slack/Email, CRMs, databases, and triggers without writing 90% of the glue. Powerful enough that many enterprise AI teams build their first dozen agentic workflows here before promoting to LangGraph.
DSPy 3 (Stanford) — the framework that lets you compile and optimise your prompts and chains using data and metrics, instead of hand-tuning. Treating prompts as code that the optimiser updates is genuinely different and increasingly common in production.

Learning objectives (n8n)

Understand n8n's nodes, triggers, and credentials.
Build an "AI Agent" node connected to OpenAI / Anthropic.
Connect MCP servers to n8n.
Self-host n8n with Docker.

Learning objectives (DSPy)

Define Signatures and pick a Module (Predict, ChainOfThought, ReAct).
Use MIPROv2 to optimise prompts with data.
Save and reload an optimised program.
Know when DSPy beats prompt engineering by hand.

1. n8n — what it is and when it wins

n8n (pronounced "n-eight-n") is a node-based workflow editor like Zapier, but:

Open source / self-hosted (Docker, license: Sustainable Use).
400+ integrations (Slack, Gmail, Notion, Postgres, OpenAI, HuggingFace, MCP, webhook, schedule, etc.).
Code nodes for arbitrary JS/Python.
AI Agent node with built-in chat memory, tool selection, and structured output.

When n8n wins:

Quick "send Slack when X happens" agent integrations.
Connecting non-AI systems (CRMs, ERPs) to your agent.
Letting non-engineers maintain workflows.
Triggering agents on cron / webhook / email.

When you outgrow n8n:

Heavy multi-agent topologies → LangGraph.
Need fine-grained evals + CI gates → LangGraph + RAGAS.
Need code-level traceability → custom Python.

Self-host with Docker

yaml

# docker-compose.yml
version: "3.8"
services:
  n8n:
    image: n8nio/n8n:latest
    ports: ["5678:5678"]
    environment:
      - N8N_BASIC_AUTH_ACTIVE=true
      - N8N_BASIC_AUTH_USER=admin
      - N8N_BASIC_AUTH_PASSWORD=changeme
      - N8N_HOST=localhost
      - WEBHOOK_TUNNEL_URL=http://localhost:5678
    volumes: ["~/.n8n:/home/node/.n8n"]

bash

docker compose up -d
# open http://localhost:5678

A first agentic workflow

Trigger: "On webhook" (POST /n8n-agent).
AI Agent node (LangChain-style) — pick OpenAI as model, add tools:
- HTTP Request tool (call any REST API).
- Postgres tool (run a parameterised query).
- Tavily community node (web search).
Structured output node — validate against a JSON schema.
Slack node — post the answer to a channel.
Wire connections, set credentials, click Active.

Every step is configurable in the UI. You can drop down to a Code node for any custom logic. The result: a real agent doing real work in 30 minutes, no boilerplate.

MCP from inside n8n

Recent versions ship an MCP Client node — point it at any MCP server (stdio or HTTP), pick the tools you want, and they show up automatically in the AI Agent node's tool list. This is enormous: the moment your team has a few MCP servers, every n8n workflow can use them.

When to embed n8n in your stack

A common 2026 pattern:

n8n orchestrates triggers, retries, multi-system fan-outs, and humans-in-the-loop.
LangGraph lives behind a webhook to do the deep agentic work.
MCP servers expose your data tools to both.

Diagram of trust:

[trigger: webhook/cron/Slack] → [n8n flow] → [LangGraph agent service] → [MCP servers]

2. DSPy — programmatic prompting

DSPy (Stanford, 2023; v3.x in 2026) reframes prompting as a programming model. You write your task as modules with signatures, supply a metric and a tiny dataset, and an optimiser searches for the best prompt + few-shot demos automatically.

The mindset shift: prompts become weights. You compile programs the way you compile neural networks.

Install

powershell

uv add dspy-ai

Signatures (input → output contract)

python

import dspy

class Sentiment(dspy.Signature):
    """Classify the sentiment of a sentence."""
    sentence: str = dspy.InputField()
    label: Literal["positive", "negative", "neutral"] = dspy.OutputField()

A signature is the spec of one LLM call. DSPy generates a structured prompt from it.

Modules

python

predict = dspy.Predict(Sentiment)              # plain
cot     = dspy.ChainOfThought(Sentiment)        # adds a reasoning field
react   = dspy.ReAct("question -> answer", tools=[search_tool, calc_tool])

Configure an LLM once:

python

dspy.configure(lm=dspy.LM(model="openai/gpt-4.1-mini"))
print(cot(sentence="Service was awful, never coming back").label)

Programs (compose modules)

python

class RAG(dspy.Module):
    def __init__(self, retriever):
        super().__init__()
        self.retrieve = retriever
        self.gen = dspy.ChainOfThought("context, question -> answer")

    def forward(self, question: str):
        ctx = self.retrieve(question, k=4)
        return self.gen(context=ctx, question=question)

Now RAG is a callable program with a typed input/output. Treat it like a function.

Optimisers (the headline feature)

You give DSPy:

A metric (function (example, pred) -> float).
A small dataset (15-200 labelled examples).
An optimiser (e.g. MIPROv2).

It searches for better instructions and few-shot demos automatically.

python

from dspy.teleprompt import MIPROv2

def is_correct(ex, pred, trace=None):
    return pred.label == ex.label

trainset = [dspy.Example(sentence=s, label=l).with_inputs("sentence") for s, l in pairs]

teleprompter = MIPROv2(metric=is_correct, auto="medium")
optimised = teleprompter.compile(cot, trainset=trainset)

print(optimised(sentence="Loved the experience.").label)
optimised.save("opt_sentiment.json")

What MIPROv2 does internally:

Bootstraps few-shot example candidates from the trainset.
Generates many candidate instructions using a stronger LLM, conditioned on your data.
Searches the (instructions × demos) space with Bayesian optimisation.
Picks the combination that maximises your metric on a held-out validation set.

Reported lift in DSPy docs: 5-46% over hand-crafted prompts on benchmarks like GSM8K. Most teams see 8-20% on real internal tasks.

Other optimisers to know in 2026:

BootstrapFewShot — quick, only optimises demos. Good for ≤20 examples.
BootstrapFewShotWithRandomSearch — better when you have 50-200 examples.
GEPA — reflective evolution; uses textual feedback in addition to scores. Very strong on hard tasks but more compute.
BootstrapFinetune — produces a fine-tuned small model from your program traces.

Reload and ship

python

loaded = dspy.ChainOfThought(Sentiment)
loaded.load("opt_sentiment.json")  # restores prompt + demos

Check the JSON into Git. Your "model improvements" are now portable artefacts.

3. When to choose DSPy vs prompt engineering

Situation	Pick
You have <10 examples and a flexible task	Hand-write the prompt.
You have ≥30 labelled examples and a measurable metric	DSPy with `MIPROv2`.
You need to switch LLMs without rewriting prompts	DSPy abstracts the LLM.
You need extreme accuracy and have explicit textual feedback	DSPy `GEPA`.
You are building one-off creative content	Prompt engineering.

DSPy is also the cleanest answer to "how do you keep prompts in version control as code?"

Hands-on labs (4 hours each)

n8n lab

Build a support-ticket triage workflow:

Trigger: webhook from your help-desk (or simulated via curl).
AI Agent node classifies into billing/technical/general/spam (use structured output).
Conditional split → respective Slack channel.
For "billing" tickets, also call your MCP billing server via the MCP client node.
HIL: send a Slack approval on actions over $X; only execute on approval.
Persist the trace into Postgres for audit.

DSPy lab

Build a classifier + RAG hybrid:

200 labelled support tickets (synthetic from an LLM).
Define a DSPy program: route_intent → answer (RAG with KB) → critique.
Metric: combined accuracy (intent) and RAGAS faithfulness on answers.
Optimise with MIPROv2(auto="medium").
Compare: hand-written prompts vs DSPy-optimised.
Save the JSON, ship in tests/dspy_program.json, load in CI to assert no regression.

Common pitfalls

n8n

Storing API keys in plain workflows — use Credentials.
Webhook URL leaks — protect with header secrets.
Heavy work in n8n — push compute to your Python service; let n8n orchestrate.
No retries on flaky external systems — n8n has retry settings per node.

DSPy

Tiny train sets — < 10 examples → optimiser overfits.
Wrong metric — DSPy optimises exactly your metric. Pick carefully.
Forgetting .with_inputs(...) — examples without it break optimisation.
Ignoring cost — auto="heavy" can run thousands of LLM calls. Monitor.

Self-check

What is the difference between an n8n trigger and an AI Agent node?
How does the n8n MCP Client node make MCP servers usable in workflows?
What does a DSPy Signature do at runtime?
Why is MIPROv2 the production default?
How do you ship an optimised DSPy program to production?

n8n workflows and DSPy — programmatic prompting

Why this matters

Learning objectives (n8n)

Learning objectives (DSPy)

1. n8n — what it is and when it wins

Self-host with Docker

A first agentic workflow

MCP from inside n8n

When to embed n8n in your stack

2. DSPy — programmatic prompting

Install

Signatures (input → output contract)

Modules

Programs (compose modules)

Optimisers (the headline feature)

Reload and ship

3. When to choose DSPy vs prompt engineering

Hands-on labs (4 hours each)

n8n lab

DSPy lab

Common pitfalls

n8n

DSPy

Self-check

References