Ollama on SwissGPU

Introduction

Ollama is one of the simplest ways to run local language models on a GPU server. It installs quickly, exposes a small HTTP API, and makes it easy to pull and test models directly from the terminal.

On a SwissGPU machine, the usual flow is to install Ollama, start the service, pull a model, confirm that it loads on the GPU, and optionally expose the API on a reachable port.

What You Need

An active SwissGPU Linux server with an NVIDIA GPU.
Working SSH access.
Enough VRAM for the model you plan to run.
A reachable port if you want to access the API directly from outside the server.

Step 1: Verify GPU Availability

After logging in by SSH, check that the GPU is visible:

nvidia-smi

Step 2: Install Ollama

curl -fsSL https://ollama.com/install.sh | sh

The official Linux installer sets up the Ollama binary and systemd service.

Step 3: Start the Service

sudo systemctl start ollama
sudo systemctl status ollama

Once the service is active, the local API is available on port 11434.

Step 4: Pull and Run a Model

Download a model first:

ollama pull gpt-oss:20b

Then launch it interactively:

ollama run gpt-oss:20b

For a quick first test, ask a short prompt and confirm that text is generated without errors.

Step 5: Confirm the Model Is Using the GPU

ollama ps

The processor column shows whether the active model is running on GPU, CPU, or split across both.

Step 6: Test the HTTP API

Ollama exposes a simple local API by default. Test it directly on the server with:

curl http://localhost:11434/api/generate -d '{
  "model": "gpt-oss:20b",
  "prompt": "Write a one-sentence hello from Ollama."
}'

Step 7: Optional Direct Network Access

If your server already has open ports and you want to reach Ollama directly, edit the systemd override:

sudo systemctl edit ollama

Add this content:

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"

Then reload and restart the service:

sudo systemctl daemon-reload
sudo systemctl restart ollama

After that, the API can be reached athttp://<server-host>:11434.

Common Commands

ollama list shows downloaded models.
ollama ps shows loaded models and where they run.
ollama pull <model> downloads a new model.
ollama run <model> starts an interactive session.

Troubleshooting

journalctl -e -u ollama

Model loads on CPU only: confirm GPU drivers with nvidia-smi and choose a model that fits in available VRAM.
API not reachable: verify the service is running and check whether you kept the default local bind or switched to0.0.0.0:11434.
Pull fails: verify outbound Internet access from the server.

SwissGPU