Run your own LLM on a VPS with Ollama, no GPU needed

How much RAM does an LLM need in Ollama?

Memory is the main limit for an LLM on a server without a GPU: the whole model is loaded into RAM, and if there is not enough, the model will not start. The Ollama README gives guidelines: 8 GB of RAM for 7B models, 16 GB for 13B and 32 GB for 33B. By default Ollama downloads quantized models (4 bits per weight), so a 7B model takes about 4-5 GB on disk. A long context adds memory use on top of that.

How slow is an LLM without a GPU?

Generation on a CPU is noticeably slower than on a GPU, and slower the bigger the model: a 3B model answers faster than a 13B one on the same server. Speed depends on the CPU, clock, memory and the model itself, so measure it on your own server (step 3) rather than trusting someone else's numbers. An LLM on a CPU makes sense when:

• you need a chatbot or assistant for a small team, where an answer in a few seconds is acceptable;
• you need to summarize, classify or tag modest volumes of text in the background - support requests or reviews, for example;
• data must not go to an external API and the model has to run on your own server;
• you want a sandbox for experimenting with prompts and models without paying per token.

Step 1. Install Ollama

First check how much RAM, how many cores and how much free disk the server has - models take gigabytes. Then install Ollama with the official script: it downloads the program, creates an ollama user and a systemd service that starts at boot.

free -h
nproc
df -h /

curl -fsSL https://ollama.com/install.sh | sh
ollama --version
sudo systemctl status ollama

The script runs with root privileges, so you may want to download and read it first. With the script install, models are stored in /usr/share/ollama/.ollama/models.

Step 2. Download and run a model

ollama run <model> downloads a model from the library at ollama.com and opens a chat in the terminal; type /bye to exit. Start with a small model such as llama3.2:3b: it downloads quickly and shows what the server can do.

ollama run llama3.2:3b
ollama list
ollama ps

ollama list shows downloaded models, ollama ps the ones loaded in memory. After 5 idle minutes Ollama unloads a model from memory, so the first answer after a pause takes longer while the model loads again. ollama rm <model> deletes models you no longer need.

Step 3. Measure generation speed

The --verbose flag prints statistics after every answer. The key line is eval rate: how many tokens per second the model generates on your server. prompt eval rate is how fast the prompt is read, load duration is the time to load the model into memory.

ollama run llama3.2:3b --verbose

Ask a few questions typical of your task and compare eval rate across model sizes - that way you pick the largest model that still answers fast enough. How to evaluate the server's CPU and disk in general is covered in VPS benchmarking.

Step 4. Call the model through the API

The Ollama service exposes an HTTP API on 127.0.0.1:11434 for scripts and bots on the same server. With "stream": false the answer arrives as one complete JSON object instead of token by token:

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.2:3b",
  "prompt": "Explain what a VPS is in two sentences",
  "stream": false
}'

The answer text is in the response field. The same way you can connect a Telegram bot on a VPS or your own app to Ollama: requests go to a local address and never leave the server.

Step 5. Outside access: an SSH tunnel instead of an open port

The Ollama API has no authentication: anyone who reaches port 11434 can use the model and load your server. So do not change the bind address to 0.0.0.0 and do not open 11434 in UFW. To reach it from your own computer, forward the port over SSH:

# on your own computer, not on the server
ssh -N -L 11434:localhost:11434 user@SERVER_IP
# while the tunnel is open, the API is available locally
curl http://localhost:11434/api/tags

Step 6. The Open WebUI web interface

Open WebUI is a ChatGPT-style web interface for Ollama models: chat history, model selection, multiple users. It runs in Docker (see installing Docker). The host-network variant sees Ollama on 127.0.0.1 and serves the interface on port 8080:

docker run -d --network=host \
  -v open-webui:/app/backend/data \
  -e OLLAMA_BASE_URL=http://127.0.0.1:11434 \
  --name open-webui --restart always \
  ghcr.io/open-webui/open-webui:main

With UFW enabled, port 8080 is closed from outside - open the interface through an SSH tunnel (ssh -N -L 8080:localhost:8080 user@SERVER_IP, then http://localhost:8080 in the browser). Create an account right away: the first one registered becomes the administrator.

Which Tihost VPS to pick for Ollama?

Tihost plans have no GPUs - models run on AMD Ryzen 9 CPUs. For 7B models, Advanced with 8-12 GB of RAM works; for 13B and 33B, Power with 16-32 GB. The Ryzen 9 9950X in Poland supports AVX-512 and DDR5 memory, which speed up neural network math on a CPU - more in Ryzen 9 5950X vs 9950X.