How to Run Ollama on Ubuntu

What Ollama Is

Ollama is a local model runner that packages the model, runtime, and command-line workflow behind a simple interface. It lets a homelab server or Ubuntu workstation download and serve large language models without sending prompts to a hosted inference provider.

The application exposes a command-line client and a local HTTP API. That makes it useful both for interactive terminal chats and as a backend for self-hosted tools, scripts, and web interfaces. Models still require substantial storage and memory, so choose a model size that fits the machine rather than assuming the largest option will run well.

Prepare the Ubuntu Host

Use a supported 64-bit Ubuntu system with enough free disk space for the models you plan to keep. The installer needs curl and may request sudo access because it installs system files and configures the Ollama service.

• Update Ubuntu and install current GPU drivers before troubleshooting acceleration.
• Leave several gigabytes of free disk space; model downloads are stored locally.
• More system RAM or GPU VRAM allows larger models and longer context windows.
• Start with a small model when validating an older mini PC or CPU-only server.

Install Ollama on Ubuntu

Run the official Linux install command in a terminal:

curl -fsSL https://ollama.com/install.sh | sh

Because piping a downloaded script to a shell grants that script execution rights, security-conscious administrators may inspect https://ollama.com/install.sh before running it. Use the official Ollama domain rather than a copied third-party installer.

The standard installer normally creates and starts a systemd service. Confirm its state with:

systemctl status ollama --no-pager
ollama -v

If the service is not running, start it and enable it at boot:

sudo systemctl enable --now ollama

Pull and Run Your First Model

The run command downloads a missing model and opens an interactive prompt. For example, start Gemma 3 with:

ollama run gemma3

The first launch can take time because Ollama must pull the model layers. Once the prompt appears, enter a question and press Enter. Use /bye to leave the session. If you prefer to separate downloading from execution, use:

ollama pull gemma3
ollama run gemma3

List locally installed models with ollama list. Model names and available variants change over time, so consult the Ollama model library when selecting a specific parameter size or quantization.

GPU Versus CPU

Ollama can run entirely on the CPU, which is convenient for repurposed homelab hardware but is generally slower. A supported GPU can accelerate generation substantially, especially when the model fits in VRAM. If only part of a model fits, performance depends on how the runtime divides work between GPU and system memory.

For NVIDIA hardware, install a compatible proprietary driver and check that the operating system can see the card with nvidia-smi. AMD acceleration on Linux uses supported ROCm hardware and drivers; support varies by GPU generation. Ollama may fall back to CPU execution when it cannot initialize a compatible accelerator, so a successful response alone does not prove GPU usage.

Verify Ollama Is Working

Keep a model session running, then open another terminal. Check active models and their processor allocation:

ollama ps

You can also verify the local API without exposing the service to your network:

curl http://localhost:11434/api/tags

A JSON response containing the local model list confirms that the API is reachable. For service problems, inspect recent logs with journalctl -e -u ollama. Avoid binding the API to a LAN interface until you understand the access controls and firewall implications.

What’s Next

After the command-line workflow is stable, connect Ollama to a local interface such as Open WebUI. That adds browser-based chats and model management while keeping Ollama as the inference backend; secure the interface before making it reachable beyond the trusted homelab network.