TOOL LAB Developer Tools Advanced Tested October 2026

Ollama: Run Private LLMs on a $20 VPS (2026)

⚡ VERDICT

Best for: Developers and consultants who need a private, self-hosted ChatGPT alternative with RAG over client documents and full data control.

Skip if: You need frontier-model quality, sub-second responses on large models, or you don't want to manage a server.

Price: Ollama is free (open source). You pay only for the VPS: ~$20/month for an 8GB RAM instance (checked 2026-10-15).

Skill level: Advanced — comfortable with SSH, Docker Compose, and Linux basics.

Time to first result: [TODO: our test] measure, don't estimate

Score: [TODO: our test] measure, don't estimate

🚀 Quick start

Provision an 8GB RAM VPS (Hetzner CX42 or equivalent) and install Docker + Docker Compose.
Deploy Ollama and Open WebUI via a single docker-compose.yml with persistent volumes.
Pull a 7B-class quantized model (q4_K_M) — larger models are unusably slow on CPU-only hardware.
Access Open WebUI on port 3000, create the admin account, and verify the model responds.

💡 Why use Ollama for this

Ollama is a free, open-source runtime for running large language models locally or on your own server. Paired with Open WebUI, it gives you a private ChatGPT-style interface with document upload (RAG), chat history, and multi-user support — all without sending a single token to a third-party API. The practical workflow: rent a small VPS, run two Docker containers, pull a quantized 7B model, and you have a private assistant for document Q&A, meeting summaries, and competitive research.

The main gotcha is hardware. Most guides promise 'powerful open-weight LLMs' without mentioning that CPU-only inference on 13B+ models is painfully slow. On an 8GB RAM VPS, stick to 3B–7B parameter models with q4_K_M quantization. Also note: Ollama itself is free, but you must budget for the VPS (~$20/month) and optionally a domain for HTTPS. Pricing and model availability checked 2026-10-15.

🛠️ How to Run Private LLMs on a $20 VPS (6 steps)

Step 1

SSH into your VPS and install Docker. Run: `curl -fsSL https://get.docker.com | sh` then `sudo usermod -aG docker $USER` and log out/in. Verify with `docker --version`.

Step 2

Create a directory `~/openwebui` and a file `docker-compose.yml` inside it. Paste this config: services for `ollama` (image: ollama/ollama, volume: ollama:/root/.ollama, port 11434) and `open-webui` (image: ghcr.io/open-webui/open-webui:main, port 3000:8080, volume: open-webui:/app/backend/data, environment OLLAMA_BASE_URL=http://ollama:11434). Add `restart: always` to both.

Step 3

Run `docker compose up -d` and wait 60 seconds. Then run `docker exec -it openwebui-ollama-1 ollama pull qwen2.5:7b-instruct-q4_K_M`. Do NOT pull llama3.1:70b or any model above 8B parameters — it will be unusably slow on CPU. The q4_K_M quantization is the sweet spot for 8GB RAM.

Step 4

Open http://YOUR_VPS_IP:3000 in your browser. Create the first admin account (this becomes the owner). Go to Settings → Models and confirm `qwen2.5:7b-instruct-q4_K_M` appears in the dropdown. Send a test message: 'In one sentence, what model are you?' to verify the connection.

Step 5

Secure the deployment with HTTPS. Install Caddy (`sudo apt install caddy`), point a domain's A record to your VPS IP, and create a Caddyfile with `yourdomain.com { reverse_proxy localhost:3000 }`. Caddy provisions SSL automatically.

Step 6

Set up weekly backups. Create a cron job that runs `docker run --rm -v openwebui_open-webui:/data -v $(pwd):/backup alpine tar czf /backup/openwebui-backup-$(date +%F).tar.gz -C /data .` and scp the tarball to your laptop.

📦 What you'll get

After completing the setup, you get a browser-based chat interface at your domain (or VPS IP) where you can create accounts, upload PDFs and text files, and chat with a private model. Document uploads are chunked and embedded locally, enabling RAG-based Q&A with citations. Chat history persists in the Open WebUI volume.

💰 Pricing (checked 2026-10-15)

Plan	Price	What you get for this workflow	Official link
Ollama (software)	`Free`	Open-source LLM runtime; no usage limits, no API fees	pricing ↗
Open WebUI (software)	`Free`	Open-source ChatGPT-style UI with RAG, multi-user, chat history	pricing ↗
Hetzner CX42 (VPS)	`~€15.59/month (~$17)`	8 vCPU, 16GB RAM, 160GB SSD — recommended for 7B models	pricing ↗
Hetzner CX32 (VPS)	`~€6.80/month (~$7.50)`	4 vCPU, 8GB RAM, 80GB SSD — minimum for 7B q4_K_M	pricing ↗
Domain + Caddy	`~$10/year + free`	Domain registration; Caddy provides automatic SSL at no cost	pricing ↗

📋 Copy-paste prompts

PROMPT 1 — Client document Q&A with citations

"I've uploaded [client name]'s contract. Answer these three questions using only the document: 1) What is the termination notice period? 2) What are the payment terms? 3) Is there a non-compete clause, and if so, what's its scope? For each answer, quote the exact clause and page number."

PROMPT 2 — Meeting transcript to action items

"Here is a 45-minute meeting transcript: [paste]. Extract: (a) every decision made, with who made it; (b) every action item, with owner and deadline if mentioned; (c) every open question that was deferred. Format as a table with columns: Type, Item, Owner, Deadline, Source quote."

PROMPT 3 — Competitive teardown from public docs

"I've uploaded three competitor pricing pages as PDFs. Build a comparison table with rows for: pricing tiers, key features per tier, overage charges, contract length, and free tier limits. Then write a 3-paragraph strategic summary of where the market is vulnerable."

⚠️ Common mistakes

❌ Pulling a 70B or even 13B model on a CPU-only VPS.

Why it happens: Every guide says 'powerful open-weight LLMs' without mentioning hardware.

✅ Quick Fix: Stick to 3B–7B parameter models with q4_K_M quantization; qwen2.5:7b-instruct-q4_K_M is the best quality/speed trade-off.

❌ Exposing port 3000 directly to the internet without HTTPS or authentication.

Why it happens: The quick-start guides skip the reverse proxy step.

✅ Quick Fix: Use Caddy with a domain name — it gives you automatic SSL and a real URL in 5 minutes.

❌ Not setting up backups before uploading client documents.

Why it happens: You're excited to use RAG and forget that the VPS can be rebuilt but your chat history can't.

✅ Quick Fix: Run the docker backup command weekly via cron and scp the tarball to your laptop.

❌ Assuming Ollama's default model is optimal for your RAM.

Why it happens: Ollama pulls a default model that may exceed your VPS memory.

✅ Quick Fix: Always specify the exact model tag with quantization, e.g., `qwen2.5:7b-instruct-q4_K_M`, and check RAM usage with `free -h`.

⚖️ Ollama vs Alternatives

Tool	Price	Best For	Verdict
	`$25/user/month (annual) or $30/user/month (monthly)`
	`Free (desktop app)`

❓ Frequently Asked Questions

Q: Can I run Ollama on a VPS without a GPU?

Yes, but only small models. On CPU-only hardware, stick to 3B–7B parameter models with q4_K_M quantization. Larger models will be too slow for interactive use.

Q: How much RAM do I need for a 7B model?

A 7B model at q4_K_M quantization uses roughly 4–5GB of RAM. An 8GB VPS is the practical minimum; 16GB gives headroom for the OS, Open WebUI, and concurrent requests.

Q: Is Ollama free?

Yes. Ollama is open-source software with no usage fees. You only pay for the hardware (VPS or your own machine) and any optional domain costs.

Q: Can I use Ollama with Open WebUI for RAG?

Yes. Open WebUI includes built-in document upload and RAG. It chunks and embeds documents locally using a default embedding model, then retrieves relevant passages when you ask questions.

Q: How do I secure my Open WebUI instance?

Never expose port 3000 directly. Use a reverse proxy like Caddy with a domain name to get automatic HTTPS. Also create a strong admin password and consider restricting sign-ups.

Q: What is the best model for an 8GB VPS?

qwen2.5:7b-instruct-q4_K_M is a strong default. Other good options include llama3.2:3b-instruct-q4_K_M for faster responses and phi3:mini for lower memory usage.

🛡️ Facts checked against official pages on October 2026:

ollama.com ↗ · github.com ↗ · www.hetzner.com ↗ · caddyserver.com ↗ · docs.docker.com ↗