Ollama vs llama.cpp vs vLLM vs LM Studio: Which Local AI Server Should You Run?

This is another entry in our Local AI Stack series. We have covered running a full local stack, turning a PC into a private AI server, and plenty of models to run on them. What we have never answered is the question that comes first: which server actually runs the model.

Search for that and you get the same list every time. Ollama, llama.cpp, LM Studio, vLLM, SGLang, ranked by tokens per second as though they were five competitors doing the same job. They are not. Understanding why makes the choice obvious in about a minute.

TL;DR

• There are two families, not five competitors. Ollama and LM Studio both embed llama.cpp, so they are close cousins. vLLM and SGLang are a different architecture entirely.
• Picking between Ollama and LM Studio is a UX decision. Picking between Ollama and vLLM is an architecture decision. Only one of those is worth agonising over.
• Everything here is MIT or Apache 2.0 except LM Studio, which is closed source but free, including at work.

The split that explains everything

Family one is built on llama.cpp. llama.cpp is the inference engine: a C++ implementation that runs GGUF-quantised models on almost anything, including CPUs and Apple Silicon. Ollama wraps it in a daemon with a model registry. LM Studio wraps it in a desktop GUI.

This is why throughput comparisons between Ollama and LM Studio are close to meaningless. Community benchmarks put them within roughly five percent of each other, and that is exactly what you would expect from two programs running the same engine underneath. The differences that matter are ergonomics and how quickly each one ships upstream llama.cpp improvements, not speed.

Family two is built for concurrency. vLLM and SGLang use continuous batching and PagedAttention. Instead of processing requests one at a time, they keep the GPU saturated by continuously slotting new requests into the batch as older ones finish, and they manage the KV cache in pages rather than contiguous blocks, which cuts the memory waste that otherwise limits how many requests fit at once.

Ollama and LM Studio do not do this. For a single user that costs you nothing, because there is only one request in flight. Under concurrent load it is the whole ballgame. Community benchmarks report vLLM at more than an order of magnitude above Ollama's concurrent throughput, and while the exact multiple depends entirely on your workload, the mechanism is not in dispute.

So the honest framing is: if you are one person on one machine, benchmark differences between these tools are noise. If you are serving multiple users, Ollama is the wrong shape, not the slower option.

Family one: the llama.cpp lineage

llama.cpp

The engine everything else in this family is built on, at 121,567 stars and MIT licensed. Note it now lives under the ggml-org organisation rather than the original ggerganov account.

Use it directly when you need control or when your hardware is unusual. It runs on CPUs, on Apple Silicon, on modest GPUs, and on things that are not really meant to run language models at all. Its quantisation format, GGUF, is the one you keep seeing on Hugging Face, and if you want to understand what those suffixes mean, we broke that down in quantization formats explained.

The cost is ergonomics. You manage model files, flags and server processes yourself.

Ollama

The most popular project in this entire comparison at 176,859 stars, MIT licensed, and the reason most people's first local model ever ran. It hides llama.cpp behind a daemon and a registry:

ollama run qwen3

That is the whole onboarding. It also exposes an OpenAI-compatible endpoint, which is why nearly every local AI tutorial, including our own n8n generator build, assumes it. It has been moving fast lately, picking up an interactive agent mode in v0.32.

Its limitation is the one above: no continuous batching. Excellent for one user, wrong tool for fifty.

LM Studio, and Jan

LM Studio is the polished desktop app: browse models, click, chat, with a local server when you want one. It bundles both llama.cpp and MLX, so it handles Apple Silicon natively.

It is also the one closed-source tool here, and that deserves a precise statement rather than a warning. LM Studio is proprietary, developed by Element Labs, but it is free for personal and commercial use. The separate work licence was removed in July 2025, so using it at your job requires no form and no fee. Its lms CLI is MIT.

So the reason to prefer an alternative is auditability, not cost. If you want a GUI and open source, Jan is the Apache 2.0 option covering the same ground. Closed source does not mean paid here, and it is worth being clear about which objection actually applies to you.

Family two: built for concurrency

vLLM

The default choice for serving, at 87,140 stars and Apache 2.0. It originated PagedAttention, supports a very wide range of model architectures, and exposes an OpenAI-compatible API so most clients work unchanged.

It is what you reach for when requests arrive in parallel. It is also what the wider ecosystem assumes: both the NemoClaw agent stack and Qwen3-ASR document vLLM as their serving path.

The tradeoff is that it wants a real GPU and more setup than ollama run. There is no CPU-only story worth having.

SGLang

The younger sibling at 30,736 stars, also Apache 2.0, also continuously batched. Its specialisation is workloads where many requests share a long common prefix, which describes RAG almost exactly: the same retrieved context and system prompt in front of many different questions. On that shape it can edge out vLLM.

For general serving with wide model support, vLLM remains the shorter path. Reach for SGLang when your traffic actually has that prefix-heavy shape rather than on principle.

The one that quietly fell behind

Hugging Face's Text Generation Inference used to appear in every list like this. It is worth checking whether it still should. Repository activity, read from the GitHub API on the date below:

Four of the five pushed commits the same day we checked. TGI last saw a push four months earlier and sits at roughly an eighth of vLLM's stars. We are not calling it dead, and Apache 2.0 code does not stop working. But if you are choosing a serving stack to build on in 2026, that row is the one to think hardest about, and the dates say more than any opinion would.

Apple Silicon

MLX is Apple's array framework built for unified memory, and mlx-lm runs language models on it. On an M-series Mac it is generally the fastest path, because it is designed for the architecture rather than ported to it.

You do not necessarily need to touch it directly. LM Studio bundles MLX alongside llama.cpp and picks appropriately, which is a large part of why it is the common recommendation for Mac users who do not want to think about any of this. We used a Mac-native path in running LTX-2 locally on Mac for the same reason.

So which one

Just starting, or one developer: Ollama. The onboarding is genuinely one command and the OpenAI-compatible endpoint means nothing you build against it is wasted if you migrate later.

You want a GUI: LM Studio if you want the most polished experience and do not mind closed source, Jan if you would rather stay open. Both are free.

Building a product with real users: vLLM. Not because Ollama is slow, but because concurrent serving is a different problem and Ollama does not attempt to solve it. Migrating later is not painful, since both speak the same API shape, so starting on Ollama and moving when you have traffic is a reasonable plan rather than a mistake.

RAG at volume: benchmark SGLang against vLLM on your own traffic before committing. The prefix-sharing advantage is real but workload-specific.

Weird hardware, or no GPU: llama.cpp, and see AirLLM for the extreme end of low-VRAM tricks.

The ten-minute move: if you have never run a local model, install Ollama and pull one. If you already have and are wondering whether to switch, the useful question is not which is faster but whether you have concurrent users. If you do not, you are already on the right tool.

Sources and further reading

• Quantization formats explained: what GGUF and the rest actually mean
• Run a full local AI stack on your own hardware
• ggml-org/llama.cpp on GitHub (MIT)
• ollama/ollama on GitHub (MIT)
• vllm-project/vllm on GitHub (Apache 2.0)
• sgl-project/sglang on GitHub (Apache 2.0)
• LM Studio: free for use at work
• Apple MLX on GitHub

Tested on: not independently tested. Our bench has no NVIDIA GPU, so we ran no throughput benchmarks and quote none as our own. The star counts, licences and last-push dates in the table above were read directly from the GitHub API on the date below and are the load-bearing facts in this article. Throughput comparisons are community-reported and workload-dependent; we describe the architectural mechanism, continuous batching and PagedAttention, rather than repeating a specific multiplier as though it were universal. LM Studio's licensing is quoted from its own announcement.
Date checked: 2026-07-25