Ollama AI: Local Runtime Specs, API, and Token Costs

Ollama operates as a local large language model inference runtime that executes open weights directly across consumer and server hardware without per-token API charges, though the hardware and power are still yours to pay for. Its daemon exposes a developer-focused REST API on port 11434 with native support for structured JSON schemas, adjustable thinking budgets, and Low-Rank Adaptation (LoRA) integration.123

For engineering teams evaluating self-hosted infrastructure against managed endpoints, Ollama pairs an accessible command-line workflow with granular generation controls. Understanding where local execution delivers cost leverage requires examining runtime architecture, configuration files, and commercial token price differentials.14

What is Ollama and its runtime architecture?

Ollama was first released on July 8, 2023, by developers Jeffrey Morgan and Michael Chiang. Built as an open-source project under the MIT License, the software is written in Go, C, and TypeScript, reaching stable release version 0.40.6 on October 16, 2026. The company raised $65 million in funding in July 2026 to support its ecosystem.1

Under the hood, Ollama uses the llama.cpp backend for local model inference across macOS, Linux, and Windows. As detailed in our breakdown of local AI model hardware requirements, quantization lowers computational and memory requirements so consumer-grade silicon can load larger parameter weights.13

Scale metrics published by Ollama report more than 9 million monthly installs, 1 billion model downloads, and over 200 trillion tokens served globally. Developers interact with the runtime via official client libraries including ollama-python and ollama-js, alongside CLI commands like ollama launch claude and integrations with agents like OpenClaw.56

Model weights supported across parameter scales

The Ollama library organizes weights ranging from lightweight edge models to multi-hundred-billion parameter foundations. Meta's Llama 3.2 family is packaged in compact 1B and 3B parameter sizes, while Llama 3.1 is distributed in 8B, 70B, and 405B parameter configurations. Alibaba's Qwen 2.5 series spans 0.5B to 72B parameter variants while handling context windows up to 128K tokens.7

For reasoning and mixture-of-experts workloads, DeepSeek-R1 is available in Ollama across seven distinct sizes: 1.5B, 7B, 8B, 14B, 32B, 70B, and 671B. Additionally, DeepSeek-V3 is supported as a 671B total parameter Mixture-of-Experts architecture that activates 37B parameters per token during inference, offering high capacity for complex pipelines.7

Controlling execution via Modelfile and the REST API

Operational customization in Ollama centers on the Modelfile. Developers use this manifest to declare a base model, configure default system prompts, lock generation parameters such as temperature, and attach LoRA adapters without altering core weights. Ollama focuses specifically on adapter-based fine-tuning rather than full weight updates.3

The daemon serves a REST API commonly exposed on port 11434. All response durations are returned in nanoseconds, and streaming can be toggled off by setting {"stream": false}. For reasoning architectures, developers can calibrate the think field using boolean flags or discrete thinking levels: low, medium, high, or max.12

Production pipelines rely on two specific API parameters. The keep_alive parameter governs how long a model stays resident in GPU or system memory after a generation, defaulting to 5 minutes (5m). Enforcing structured outputs occurs by passing a JSON schema into the format parameter, guaranteeing downstream validation.2

Teams measuring generation velocity calculate throughput in tokens per second directly from API metadata. Dividing eval_count by eval_duration and multiplying by 10^9 extracts exact token generation speed, allowing teams to compare local performance against hosted targets like vLLM concurrency benchmarks.2

Port 11434 exposure and network security

The convenience of Ollama's local setup introduces operational vulnerabilities when run on shared infrastructure. In January 2026, security researchers at SentinelOne and Censys found numerous Ollama servers exposed to the public internet because instances bound directly to the INADDR_ANY interface (0.0.0.0).1

Because the default REST API on port 11434 lacks native authentication layers, exposing the port permits unauthorized model pulls, unrestricted generation loops, and resource exhaustion. Deployments must restrict interfaces to localhost or sit behind reverse proxies enforcing authentication, as documented in our guide to self-hosted AI agent runtimes.1

Self-hosted weights versus commercial API pricing

Deciding between local Ollama instances and commercial endpoints requires evaluating unit token economics. Proprietary frontier models enforce steep pricing tiers. Under standard processing, OpenAI GPT-6 Astra costs $10.00 per 1M input tokens, $1.00 per 1M cached input tokens, and $50.00 per 1M output tokens, with web search tool retrieval billed at $10.00 per 1k calls while search content tokens remain free.8

OpenAI's mid-tier GPT-6.1 Sol costs $2.00 per 1M input tokens, $0.10 per 1M cached inputs, and $10.00 per 1M output tokens. At the budget tier, GPT-6 Luna runs $0.10 per 1M input tokens, $0.01 per 1M cached inputs, and $0.50 per 1M outputs for context lengths under 272K. OpenAI's Batch API cuts input and output fees by 50% for asynchronous workloads completed across 24 hours.8

According to data from our LLM API pricing timeline, combining 1 million input and 1 million output tokens on OpenAI gpt-6-luna totals $0.60, whereas gpt-6-astra totals $60, representing a 100-fold cost increase. Open-weight DeepSeek models hosted on Alibaba Cloud, as captured in our dataset, charge $0.574 input and $2.294 output per 1M tokens for Qwen deepseek-r1, and $0.287 input and $1.147 output per 1M tokens for Qwen deepseek-v3.4

Run Ollama locally when your architecture needs high-volume batch processing with no per-token fees, strict on-premise data isolation, or fine-tuned LoRA adapters running on owned silicon. Shift to commercial endpoints when workloads exceed local VRAM limits or demand frontier capabilities that cannot fit on internal infrastructure without cluster overhead.38