Run Local LLMs with n8n and Ollama (Private AI, No API Costs)
· By AI Compare
Private AI workflows with no per-token costs: connect n8n to local models with Ollama.
Why run local LLMs with n8n
Ollama runs open-weight models (Llama, Qwen, Mistral, Gemma and others) on your own machine or server. Connect it to n8n and you get AI workflows where:
- No data leaves your infrastructure, useful for confidential documents and client data.
- No per-token API costs; you pay only for hardware.
- No rate limits from external providers.
The trade-off: local models are usually less capable than the best cloud models, and you need enough RAM (and ideally a GPU).
Hardware guide
| Model size | RAM needed (approx.) | Good for |
|---|---|---|
| 1–4B parameters | 4–8 GB | Classification, extraction, short summaries |
| 7–9B | 8–16 GB | General chat, drafting, simple agents |
| 14B+ | 16–32 GB+, GPU recommended | Better reasoning, longer documents |
A small VPS without a GPU will run small models slowly. For anything beyond light use, use a machine with a GPU or a Mac with Apple Silicon.
Step 1: Install Ollama and pull a model
Install Ollama from ollama.com, then:
ollama pull llama3.1:8b
ollama run llama3.1:8b "Hello"
Ollama serves an API on port 11434.
Step 2: Connect n8n to Ollama
In n8n, create an Ollama credential with the base URL:
| Setup | Base URL |
|---|---|
| n8n and Ollama installed directly on the same machine | http://localhost:11434 |
| n8n in Docker, Ollama on the host | http://host.docker.internal:11434 (on Linux add --add-host=host.docker.internal:host-gateway) |
| Both in the same Docker Compose project | http://ollama:11434 (the service name) |
The most common error is using localhost from inside the n8n container, which points to the container itself, not your computer.
Step 3: Build the chat workflow
- Chat Trigger: gives you a chat window in n8n (and an optional public chat URL).
- AI Agent or Basic LLM Chain.
- Ollama Chat Model sub-node: choose the model you pulled.
- Optional Simple Memory for multi-turn chat.
Open the chat panel and send a message. The first response may take longer while the model loads into memory.
Use cases that work well locally
- Summarising internal documents and meeting notes.
- Classifying support tickets or emails.
- Extracting fields from invoices or contracts into JSON.
- Private RAG over company files using Embeddings Ollama and a local vector store (Qdrant or Postgres with pgvector).
Tips
- Smaller models follow tool-calling instructions less reliably. Keep agents to one or two tools, or use a Basic LLM Chain with structured output.
- Set temperature low (0–0.3) for extraction and classification.
- n8n's "self-hosted AI starter kit" bundles n8n, Ollama, Qdrant and Postgres in Docker Compose, a fast way to get everything running together.
