Ollama as a local network service: deployment and integration

Ollama runs language models locally on a GPU. This guide covers a deployment architecture built as a network service: one central Ollama server that several clients reach over HTTP. The approach makes better use of the GPU, removes redundant installations, and keeps model management in one place.

Instead of maintaining an isolated copy of Ollama on every workstation, this architecture centralises the service on a machine with a GPU. Clients reach the models through the network API.

What follows covers configuration, validation and tuning for a working proof of concept, including the problems you are likely to hit and what to do about them.

Understanding the network architecture

Before diving in, let us picture what we are building.

Workstation 1 → [Ollama + local GPU] fast, consumes resources
Workstation 2 → [Ollama + local GPU] fast, expensive, wasteful
Workstation 3 → [Ollama + local GPU] fast, expensive, wasteful

Server → [Ollama + RTX 3080 GPU] ← Workstation 1
                                   ← Workstation 2
                                   ← Workstation 3
Everyone connects over HTTP. One GPU, one installation.

The architecture is simple: a machine with a GPU runs Ollama listening on port 11434, reachable on the network, and each client workstation connects over HTTP.

Upsides

• Centralisation: one installation, one upgrade
• Cost: the GPU is shared between several users
• Flexibility: switch workstations without reinstalling anything
• Performance: a dedicated GPU for everyone

Downsides

• Minor network latency (negligible on a local network)
• A dependency on the server

Prerequisite: installing the NVIDIA drivers

This is trap number one. Without a GPU driver, Ollama runs on the CPU and becomes unusable.

Checking that a GPU is present

Connect to your server and run:

ssh [email protected]
lspci | grep -i nvidia

Expected output:

01:00.0 VGA compatible controller: NVIDIA Corporation
GA102 [GeForce RTX 3080]

If nothing comes back, check that the GPU is physically installed and enabled in the BIOS.

Installing the driver

sudo apt update
sudo apt install nvidia-driver-550 -y
sudo reboot

After the reboot, check:

nvidia-smi

You should see a table with your GPU, its VRAM, its temperature and so on. If you do, you are good to go.

Installing and configuring Ollama for the network

Step 1: basic installation

curl -fsSL https://ollama.com/install.sh | sh

Check it:

which ollama
# /usr/local/bin/ollama

Step 2: network configuration (the key part)

By default, Ollama only listens on 127.0.0.1:11434, the local machine. To open it to the network, you have to change that.

Create a systemd override:

sudo systemctl edit ollama

The editor opens with two ### blocks around an empty section. Add this between them:

[Service]
Environment="OLLAMA_HOST=0.0.0.0"

Save (Ctrl+X, Y, Enter in nano).

Step 3: starting the service

sudo systemctl daemon-reload
sudo systemctl enable ollama
sudo systemctl start ollama

Check it:

sudo systemctl status ollama
ss -tlnp | grep 11434

Expected output:

● ollama.service - Ollama Service
     Active: active (running)

LISTEN 0      4096               *:11434            *:*

The *:11434, rather than 127.0.0.1:11434, means Ollama is listening on every network interface. That is what we want.

Pulling and testing models

Recommended models

For quick prototyping (lightweight):

ollama pull qwen2.5:0.5b

• Size: 755 MB
• Response time: around 2 to 5 seconds
• VRAM usage: around 2 GB

For code and complex tasks (recommended):

ollama pull qwen2.5-coder:7b

• Size: 4.9 GB
• Response time: around 10 to 20 seconds
• VRAM usage: around 6 GB (on an RTX 3080)

Testing from a client machine

From another machine on your network:

curl http://192.168.0.240:11434/api/generate -d '{
  "model": "qwen2.5-coder:7b",
  "prompt": "Write a Python function that computes the Fibonacci sequence",
  "stream": false
}'

Expected output:

{
  "model": "qwen2.5-coder:7b",
  "response": "def fibonacci(n):\n    if n <= 1:\n        return n...",
  "done": true
}

If that works, Ollama is reachable from the network.

Checking GPU usage

On the server, make sure the models really run on the GPU:

ollama ps

NAME                ID              SIZE      PROCESSOR    CONTEXT
qwen2.5-coder:7b    dae161e27b0e    5.2 GB    100% GPU     8192

NAME                ID              SIZE      PROCESSOR    CONTEXT
qwen2.5-coder:7b    dae161e27b0e    4.9 GB    100% CPU     8957

Integrating with Aider (the recommended CLI)

Aider is a CLI client for generating code with Ollama. It is the best option for local development.

Installation

pip install aider-chat

Configuration

Start Aider against your Ollama server:

OLLAMA_API_BASE=http://192.168.0.240:11434 \
aider --model ollama/qwen2.5-coder:7b

Or, if aider is not on your PATH:

OLLAMA_API_BASE=http://192.168.0.240:11434 \
~/.local/bin/aider --model ollama/qwen2.5-coder:7b

Usage

aider> Add a function that validates email addresses with regex

aider> Create unit tests for the email validator

aider> Explain the performance implications of this function

Tip: create an alias so you do not retype the whole command:

# In ~/.bashrc or ~/.zshrc
alias aider-ollama="OLLAMA_API_BASE=http://192.168.0.240:11434 \
aider --model ollama/qwen2.5-coder:7b"

Then simply:

aider-ollama

Common troubleshooting

Problem: connection timeout from the client

curl: (7) Failed to connect to 192.168.0.240
port 11434: Connection refused

Diagnosis:

1. Check that the Ollama server is running: sudo systemctl status ollama
2. Check that port 11434 is open: ss -tlnp | grep 11434
3. Check the network connection: ping 192.168.0.240

Problem: the GPU is not used (100% CPU)

This means the NVIDIA drivers are not being detected.

nvidia-smi

If you get an error, reinstall the drivers:

sudo apt install --reinstall nvidia-driver-550 -y
sudo reboot

Problem: not enough VRAM

With less than 6 GB of free VRAM, Ollama cannot load the 7B model.

Ways out:

1. Use a smaller model: qwen2.5:0.5b (755 MB)
2. Free up VRAM by closing demanding applications
3. Use quantization: ollama pull qwen2.5-coder:7b-q4 (4-bit, around 2 GB)

Problem: the model stays in memory after use

When you are tired of spending VRAM on nothing:

# List loaded models
ollama ps
# Stop a model
ollama stop <model_name>

Real use cases and results

Test 1: simple code generation

Prompt:

Write a Python function that adds two numbers

Results:

• Response time: around 12 seconds
• Tokens: 618 sent, 124 received
• Quality: correct code, docstring included

Test 2: complex generation with memoisation

Prompt:

Create a function that computes a factorial
with memoisation and error handling

Results:

• Response time: around 18 seconds
• Tokens: 812 sent, 249 received
• Quality: correct error handling, docstring, usage examples

GPU benchmark

Server machine:

• OS: Ubuntu 22.04
• GPU: NVIDIA RTX 3080 (10 GB VRAM)
• Driver: NVIDIA 550.163.01
• CUDA: 12.4

GPU usage under load:

VRAM used: 6031 MiB / 10240 MiB
(58.9%)
Temperature: 45°C
Power: 24W / 340W
Ollama process: 100% GPU

Excellent performance, no thermal throttling.

Limitations and alternatives

Limitation: Claude Code Router is not compatible

As of December 2025, Claude Code forces the extended thinking mode, which Ollama does not support. Claude Code Router is stuck on that limitation. For Claude Code, use Aider instead.

Limitation: network latency

Even on a local network there is a small latency, roughly 50 to 100 ms compared to a local call. Negligible for development work.

Going further: a Claude Code-like interface

A complete suite called ollama-code is available for download. It automates the whole configuration and ships ready-to-use scripts.

Installing the package (three steps)

1. Extract and install

unzip ollama-code-package.zip
cd ollama-code-package
bash bin/ollama-code-setup

2. Check the installation

ollama-code --help

3. Use it in a project

ollama-code ~/my-project
# Or with the alias
code ~/my-project

What you get

Automated scripts:

• ollama-code - start Aider in a project with automatic configuration
• ollama-code-setup - full configuration in a single command
• ollama-code-demo - an interactive tour of what it does

Central configuration:

• One configuration for every project
• Automatic creation of a per-project .aider.conf.yml
• Sensible ignore patterns
• Built-in code guidelines

Main commands, inside Aider:

/map               # Codebase structure
/add file.py       # Add a file
/ask               # Question mode
/code              # Edit mode (default)
/diff              # See the changes
/commit            # Commit them
/help              # Every command

Workflows with ollama-code

Adding a feature:

$ ollama-code .
> /add src/main.py
> Add a function that computes the greatest common divisor
> /diff
> /commit

Reading code:

$ ollama-code --ask .
> Explain the architecture of this project
> /code
> Improve the error handling

Deployment checklist

Conclusion

Running Ollama as a network service turns a single GPU into a resource shared across your local network.

Results from the proof of concept: a fully working deployment in two hours. Response times of 15 to 20 seconds are acceptable for code generation.