Where Ollama Stores Models: Storage, Memory and Architecture

Ollama executes open-weight large language models locally through an inference backend based on Georgi Gerganov's llama.cpp project, distributing the platform as open-source software under the MIT License. Storage is organized into distinct content-addressed directories, separating metadata manifests from raw model weight blobs so that copied models share underlying weights without doubling disk usage. Our open model tracker lists recent open-weight releases with sizes and licences.123

Ollama Storage Architecture: Manifests, Blobs, and Deduplication

Ollama structures its local storage inside the ~/.ollama directory by decoupling structural metadata from raw tensor weights. Manifests are located at ~/.ollama/models/manifests and contain configuration files, parameter definitions, and references to associated layers. Binary model weights reside at ~/.ollama/models/blobs as content-addressed files mapped to individual sha256 digests.3

When executing ollama pull against a specific tag, Ollama retrieves the manifest, downloads the required content-addressed weight blobs, verifies each sha256 digest, and finalizes the manifest on disk. Subsequent updates initiated by re-running ollama pull inspect the remote manifest and avoid redownloading layers that already exist locally. Because weight storage is content-addressed, multiple model variants that rely on the same underlying base weights share identical blob files automatically. If Ollama does not fit your setup, compare Ollama alternatives.3

The ollama cp command uses this content-addressed design to duplicate a model under a new reference tag without doubling disk consumption. Because both manifests reference the exact same underlying blobs, disk usage remains flat until separate weights are added. When removing models, the command ollama rm targets specific tags or model names to clean up unused references and free local storage space.3

Hardware Demands: RAM Thresholds, Context Sizing, and GGUF Quantization

Deploying models on CPU-only infrastructure establishes strict baseline memory requirements determined by total parameter scale. Running a 7B parameter model on a CPU requires a minimum of 16GB RAM, while 13B models carry a recommended threshold of 32GB RAM. Larger models scaling up to 70B parameters require 64GB+ RAM to initialize and execute inference without running out of memory.4

Ollama reduces memory usage during local execution by leveraging quantized models in GGUF formats. Model tags specify parameter volume alongside quantization designations such as q2_K, q3_K_S, q4_0, q4_K_M, q5_K_M, q8_0, or unquantized fp16, alongside functional variant tags like instruct, chat, code, or text. Shorter context lengths improve inference speed and lower memory footprints during generation tasks.34

Supported Model Architectures: Dense Weights to Multimodal MoE

The Ollama library provides dense and Mixture-of-Experts architectures across varying context lengths and training volumes. Meta provides Llama 3.1 in 8B, 70B, and 405B parameter sizes with native tool support. Alibaba's Qwen2.5 series offers context lengths reaching up to 128K tokens after pretraining on datasets encompassing up to 18 trillion tokens, while Mistral Nemo delivers a 12B parameter architecture built in collaboration with NVIDIA that also provides a 128k context length.5

For extreme parameter configurations, a DeepSeek Mixture-of-Experts model implements a design containing 671B total parameters while activating only 37B parameters for each token. Google's Gemma 4 can be initiated directly from the CLI via ollama run gemma4. Next-generation multimodal workflows are handled by Llama 4 MoE architectures that accept both text and image inputs. Llama 4 Scout contains 109B total parameters with 17B active parameters, and Llama 4 Maverick contains 400B total parameters with 17B active parameters.156

Local disk footprints for these models reflect their total structural scale. The llama4:16x17b variant uses 67GB of disk space while offering a 10M context window. The larger llama4:128x17b variant consumes 245GB of disk space while providing a 1M context window.6

Managing the Model Lifecycle: Pulling, Inspecting, and Monitoring via the CLI

Managing active models requires tracking memory allocations and runtime layer placement. The ollama ps command displays all currently loaded models, their allocated memory size, processor split (such as 100% GPU allocation), and the remaining time until the model is unloaded from host memory. This allows operators to verify whether model weights remain resident in GPU VRAM or have spilled over into system RAM.3

Detailed metadata inspection is handled through the ollama show command. Operators can append flags to isolate specific components: --modelfile reveals the base template and parameter configurations, --system extracts the defined system prompt, --template shows prompt templating syntax, and --license displays the governing license information.3

Customizing and Running Models: Modelfiles, Parameters, and API Endpoints

Custom models in Ollama are constructed using a Modelfile. The configuration begins with a FROM instruction targeting a base model such as llama3.2:3b, followed by a SYSTEM instruction defining the prompt that directs model behavior. Models can be executed directly from the CLI with runtime sampling arguments, including flags like --temperature and --top-p passed directly to the ollama run command.37

The Ollama server runs a local HTTP interface on default port 11434. The REST API exposes the /api/generate endpoint, which processes JSON requests specifying the model name, text prompt, and stream preferences. Conversational tasks use the /api/chat endpoint located at http://localhost:11434/api/chat. Official client SDKs are distributed via the ollama-python and ollama-js packages.17

Installation is automated across major operating systems. Linux and macOS systems install Ollama by streaming curl -fsSL https://ollama.com/install.sh into sh, while Windows environments invoke irm https://ollama.com/install.ps1 piped to iex in PowerShell. Deployment via containerization uses the official ollama/ollama Docker Hub image. Pre-built developer tooling integrations supported by ollama launch include Claude Code, Codex, Copilot CLI, DeepSeek Harness, Droid, OpenCode, and OpenClaw via ollama launch openclaw.1

Ollama REST API: Endpoints, Parameters, and Telemetry

External applications communicate with the local server over HTTP via port 11434. Ollama provides first-party client libraries for Python and JavaScript under ollama-python and ollama-js. Direct HTTP requests target two primary endpoints: POST /api/generate for discrete completions and POST /api/chat for multi-turn conversational payloads containing a messages array.8910

Endpoints stream JSON tokens by default. Supplying stream set to false stops streaming and returns a single JSON object once processing finishes. For raw completion calls, passing raw set to true disables prompt formatting templates, sending input tokens directly to the backend.10

Advanced parameters grant granular control over resource management and output formats. The keep_alive parameter controls how long model weights remain loaded in system or GPU memory after an API request, defaulting to 5 minutes. Passing a value to keep_alive prevents continuous model reloading latency between frequent automated tasks.10

Output formatting can be constrained at runtime. Setting format to 'json' activates JSON mode, forcing output to form a valid JSON object. Alternatively, developers can supply an explicit JSON schema inside the format parameter to guarantee that generated outputs conform strictly to predefined field types. For multimodal models, the images parameter accepts a list of base64-encoded images. Thinking models also accept a think parameter, configured as a boolean or set to 'low', 'medium', 'high', or 'max'.10

The final JSON response object returned by POST /api/generate includes telemetry measuring compute stages. All duration metrics are reported in nanoseconds. The payload provides four discrete timing attributes: total_duration for the complete request lifecycle, load_duration for loading model weights into memory, prompt_eval_duration for processing context, and eval_duration for token generation.10

Generation speed is calculated from telemetry fields using exact token counts. Dividing eval_count by eval_duration and multiplying the result by 10^9 produces generation throughput in tokens per second. Monitoring this value exposes when resource constraints cause GPU execution to degrade into slower system RAM.10