Skip to content

🚀 Getting started with Ollama


Prerequisites

  • macOS (Apple Silicon recommended — since Ollama 0.40, October 2026, compatible architectures run on Apple’s MLX engine by default1) or Linux (NVIDIA or AMD GPU, or CPU only)
  • Windows: supported via WSL2 or native installer — GPU performance requires CUDA or ROCm drivers
  • At least 8 GB RAM (16+ recommended for a comfortable 7B/8B)
  • Disk space: 5–50 GB depending on the downloaded model

Installation

macOS / Linux

Fenêtre de terminal
curl -fsSL https://ollama.com/install.sh | sh

Ollama installs a system service that starts automatically at boot.

Verification:

Fenêtre de terminal
ollama --version
# ollama version is 0.40.x (0.40.16 as of 2026-11-09)
# Is the service running?
curl http://localhost:11434/
# Ollama is running

Windows

Download the installer from ollama.com/download. The installer configures the background service and adds ollama to PATH.


First model

Fenêtre de terminal
# Download and run a small recent model (~6.5 GB in Q4_K_M)
ollama run qwen3.5:9b
# Or a very light 3B (~2 GB)
ollama run llama3.2
# Or a smaller model for a quick test (~2.5 GB)
ollama run phi4-mini
# Or a coding model (~18.6 GB)
ollama run qwen3-coder:30b
# qwen2.5-coder:14b (~9 GB) is still available but is flagged as legacy by Ollama ≥ 0.32

The first run downloads the model from ollama.com/library (sizes read from the registry on 2026-10-093). Later runs use the local cache.

To exit the interactive session: /bye or Ctrl+D.


Essential commands

Fenêtre de terminal
# List downloaded models
ollama list
# See models available online
# → https://ollama.com/library
# Download without launching
ollama pull qwen3.6:35b # ~22.6 GB
# Remove a model from cache
ollama rm llama3.2
# See running processes
ollama ps
# Service logs (Linux systemd) — there is no `ollama logs` subcommand
journalctl -u ollama --no-pager --follow --pager-end
# macOS: cat ~/.ollama/logs/server.log

Per-platform log locations are described in Ollama’s troubleshooting page4.

Since Ollama 0.40.2 (October 2026), models downloaded with an earlier version are rewritten in the background on first launch (originals kept as backups); rolling back to a version < 0.40 forces you to re-pull the models — on an appliance, pin the Ollama version5.


OpenAI-compatible API

Ollama exposes a REST API on http://localhost:11434 compatible with the OpenAI format. Any library that uses the OpenAI API works without changes by switching the base URL.

Direct request

Fenêtre de terminal
curl http://localhost:11434/api/generate \
-d '{
"model": "llama3.2",
"prompt": "Explain KV Cache in 3 sentences.",
"stream": false
}'

OpenAI-compatible format (/v1/chat/completions)

Fenêtre de terminal
curl http://localhost:11434/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "llama3.2",
"messages": [
{"role": "system", "content": "You are a technical assistant."},
{"role": "user", "content": "What is the difference between prefill and decoding?"}
]
}'

Python (openai SDK)

from openai import OpenAI
client = OpenAI(
base_url="http://localhost:11434/v1",
api_key="ollama", # arbitrary value, Ollama does not verify the key
)
response = client.chat.completions.create(
model="llama3.2",
messages=[
{"role": "user", "content": "Summarize unified memory in 2 sentences."}
]
)
print(response.choices[0].message.content)

Useful settings

Longer context

By default, the Ollama server uses a 4,096-token window (OLLAMA_CONTEXT_LENGTH); the desktop app picks a value based on VRAM (4k under 24 GiB, 32k between 24 and 48 GiB, 256k above)6. To extend:

Fenêtre de terminal
# For the whole server (environment variable)
OLLAMA_CONTEXT_LENGTH=8192 ollama serve
# Via API (per request)
curl http://localhost:11434/api/generate \
-d '{"model": "llama3.2", "prompt": "...", "options": {"num_ctx": 8192}}'
# Via Modelfile (persistent)
ollama show llama3.2 --modelfile > Modelfile
# Add to the Modelfile:
# PARAMETER num_ctx 8192
ollama create llama3.2-8k -f Modelfile

Concurrent requests

By default, each model handles only one request at a time (OLLAMA_NUM_PARALLEL=1): the second one waits in the queue. Raising this value multiplies the memory reserved for context (parallelism × OLLAMA_CONTEXT_LENGTH); OLLAMA_MAX_LOADED_MODELS (3 × number of GPUs by default) caps the number of models loaded simultaneously6. For several regular users, see ⚙️ Inference engines.

Temperature and generation parameters

Fenêtre de terminal
curl http://localhost:11434/api/generate \
-d '{
"model": "llama3.2",
"prompt": "...",
"options": {
"temperature": 0.1,
"top_p": 0.9,
"num_predict": 512
}
}'

temperature sets how much randomness the model uses (0 = deterministic answers, 1 = more creative) and num_predict the maximum number of tokens generated. JSON does not accept comments: do not paste # into the request body.

Expose Ollama on the local network

By default, Ollama listens only on localhost. To make it reachable from other machines:

Fenêtre de terminal
# Linux: service environment variable
OLLAMA_HOST=0.0.0.0 ollama serve
# Or via systemd (edit /etc/systemd/system/ollama.service)
# Environment="OLLAMA_HOST=0.0.0.0"

Check performance

Fenêtre de terminal
# Run a request and measure throughput
curl http://localhost:11434/api/generate \
-d '{"model": "llama3.2", "prompt": "Say hello in 10 languages.", "stream": false}' \
| python3 -c "import sys,json; r=json.load(sys.stdin); \
print(f\"Duration: {r['total_duration']/1e9:.1f}s | \
Tokens generated: {r['eval_count']} | \
Throughput: {r['eval_count']/(r['eval_duration']/1e9):.1f} tok/s\")"

Indicative orders of magnitude (community measurements on llama.cpp, mid-2026, not individually sourced; on Apple Silicon, Ollama ≥ 0.40 uses MLX by default and throughputs may differ — measure with the command above and the protocol in 🧪 Evaluate a local model). The Apple rows cover the previous generation, as no measurements have been published for the Mac Studio M5 (theoretical upper bounds in 🏎️ Memory Bandwidth):

Hardware8B Q4 model70B Q4 model
MacBook Pro M4 Pro 48 GB (2024 generation, 273 GB/s)~30–50 tok/s✗ (~40 GB of weights > ~36 GB allocatable to the GPU by default; upper bound ≤ 6.8 tok/s even after sysctl)
Mac Studio M3 Ultra 256 GB (2025 generation, 819 GB/s)~50–70 tok/s~12–18 tok/s
AMD Ryzen AI Max PRO (192 GB, 273 GB/s)~25–35 tok/s~4–6 tok/s
RTX 4090 (24 GB VRAM)~50–80 tok/sPartial offloading
CPU only (no GPU)~3–8 tok/s< 2 tok/s

Next steps


Sources and references

Footnotes

  1. Ollama, Release v0.40.0 (“Models run on MLX on Apple Silicon by default”), October 2026. https://github.com/ollama/ollama/releases/tag/v0.40.0 ↩

  2. Ollama, Release v0.32.0 (ollama with no argument launches an agent, default entry glm-5.2:cloud; deprecation warning for legacy agent models), 11 July 2026 · Ollama, Release v0.34.2 (“first-run setup … with options to sign in or continue locally”), 15 September 2026. https://github.com/ollama/ollama/releases/tag/v0.32.0 · https://github.com/ollama/ollama/releases/tag/v0.34.2 ↩

  3. Ollama, Library and registry.ollama.ai registry (manifests: llama3.2 = 3B, 2.02 GB; qwen3.5:9b ≈ 6.5 GB; qwen3-coder:30b ≈ 18.6 GB; qwen3.6:35b ≈ 22.6 GB; qwen2.5-coder:14b ≈ 9 GB), accessed 2026-10-09. https://ollama.com/library ↩

  4. Ollama, Troubleshooting (log locations: journalctl -u ollama, ~/.ollama/logs/server.log), accessed 2026-10-09 · ollama/ollama repository, cmd/cmd.go (subcommand list, no logs). https://docs.ollama.com/troubleshooting · https://github.com/ollama/ollama ↩

  5. Ollama, Release v0.40.2 (models “upgraded in the background the first time you run them”, backups kept, re-pull required when rolling back to < 0.40), 8 October 2026. https://github.com/ollama/ollama/releases/tag/v0.40.2 ↩

  6. Ollama, FAQ (“By default, Ollama uses a context window size of 4096 tokens”, OLLAMA_CONTEXT_LENGTH, OLLAMA_NUM_PARALLEL = 1, OLLAMA_MAX_LOADED_MODELS = 3 × GPU) and Context length (app defaults: 4k / 32k / 256k depending on VRAM), accessed 2026-10-10. https://docs.ollama.com/faq · https://docs.ollama.com/context-length ↩ ↩2

  7. VulnCheck, Ollama 0.30.0 through 0.33.2 SSRF via Cross-Host Tensor Blob Redirect (CVE-2026-85180, CVSS 4.0 8.7, unauthenticated, no fixed version named), 2026-09-03 · Ollama, PR #18512 “x/transfer, server: tighten redirect handling for registry requests” (validation of the redirect target for registry and blob downloads; commit of 2026-09-17, shipped in 0.34.2). https://www.vulncheck.com/advisories/ollama-0.30.0-through-0.33.2-ssrf-via-cross-host-tensor-blob-redirect · https://github.com/ollama/ollama/pull/18512 ↩ ↩2

  8. MITRE, CVE-2026-102697 (Ollama 0.14.0 → < 0.31.2, bypass of agent-mode Bash command approval, CVSS v3.1 7.8), published 2026-09-29 · Ollama, Release v0.31.2 (“Hardened GGUF model creation”), 6 July 2026. https://cveawg.mitre.org/api/cve/CVE-2026-102697 · https://github.com/ollama/ollama/releases/tag/v0.31.2 ↩

Give my opinion

Everything is optional: one answer is enough. This site sends nothing: you send it, the way you choose.

Your noteNo note
Your opinion is about
About what? (several choices possible)
What will be sent
Send by e-mail

Private, no account: opens your mail program with the message ready, for ia-on-prem@damien.becherini.fr.

Open a GitHub issue ↗

Public, with a GitHub account. You read the issue again before you publish it.

To paste it wherever you want.

Give at least a note, a subject or a message to be able to send.