How we test and estimate

We can't run every model on every GPU, and nobody else has either. So we combine measured results from public sources (108 results from 9sources so far) with an estimate model calibrated against them. Estimates are always marked: a tilde (~) before the number, "est." in labels, and hatched bars in charts.

Memory: will it fit?

A model needs memory for three things. We add them up for every GPU and model in our database.

1. Weights

When we know the real GGUF file size for a quantization, we use it. Otherwise we use parameters × bits per weight ÷ 8, with these average bits per weight for llama.cpp's common formats:

Models released natively in a low-precision format, such as gpt-oss in MXFP4, use their published file size.

2. KV cache

Each token in the context stores a key and a value for every layer: 2 × layers × KV heads × head size × 2 bytes (16-bit). We read these values from each model's published configuration. For architectures that store less, such as DeepSeek's multi-head latent attention or models that mix sliding-window and global attention, we use a model-specific figure instead. Unless stated otherwise, figures assume 8K tokens of context and a 16-bit KV cache.

3. Runtime overhead

Compute buffers and the CUDA, ROCm, or Metal context: we budget 0.6GB plus 3% of the weights.

Verdicts

• Runs well: the total uses no more than 88% of the GPU's memory, leaving room for longer prompts and the desktop.
• Fits, tight: the total fits, but with less than 12% headroom.
• Partial offload: it doesn't fit in VRAM, but fits in VRAM plus system RAM (we assume 64GB, of which 75% is usable). It will run, much more slowly.
• Won't fit: not even with offloading.

On unified-memory systems (Apple silicon, AMD Strix Halo, NVIDIA DGX Spark) the GPU can't use all of the memory. We use the amount the GPU can allocate by default, roughly 65–75% on Macs. You can raise the Mac limit with sudo sysctl iogpu.wired_limit_mb, which lets larger models fit. For multi-GPU setups we use 97% of the combined memory to allow for uneven splits.

Speed: how fast will it run?

Generating text is limited by memory bandwidth. For every token, the GPU reads every active weight from memory once. So the time per token is roughly the bytes read divided by the bandwidth, plus a small fixed cost for launching the work:

tokens/sec ≈ 1 ÷ (active weight bytes ÷ (bandwidth × efficiency) + fixed overhead)

• Efficiency is the share of peak bandwidth real runtimes achieve. We calibrated it per platform against published llama.cpp results: about 0.80 for recent NVIDIA GPUs, 0.72 for Ampere (RTX 30-series), 0.62 for older NVIDIA cards, 0.65 for AMD Radeon, 0.55 for Intel Arc, and 0.55–0.75 for unified-memory systems.
• Mixture-of-experts models only read their active experts for each token, so they are much faster than their total size suggests. Routing makes them less efficient per byte, which we account for.
• CPU offload: weights in system RAM are read at about 45 GB/s (dual-channel DDR5 at 75% efficiency), which is why offloading a dense model is so slow.
• Multiple GPUs: with layer splitting, the cards take turns, so speed is close to one card's speed, minus some overhead per card.

The model covers single-user generation speed, which is what you feel in chat. It doesn't predict prompt processing speed (which depends mostly on compute), batched throughput for many users, or speed-ups from speculative decoding.

Recommendations

Each model has a capability score from 0 to 100, based on public benchmarks and its standing among people who run models locally. Each model's page lists the basis for its score. To pick the best models for a GPU, we start from that score and subtract points for low-precision quantization (Q3), tight fits, CPU offloading, and slow speeds (under 15, 10, and 5 tokens/sec). The highest score wins.

Measured benchmarks

We only add a measured result when it has:

• a public source we can link to,
• the exact GPU, model, quantization, and runtime, and
• single-user generation speed (llama.cpp's tg128 or equivalent), reported separately from prompt processing.

We record the runtime version and date, because llama.cpp and driver updates can change speeds by 10–30%. Where measured and estimated numbers disagree, the measured one wins on the page.

How good is the estimate model?

We check the model against every measured single-GPU result in our database each time the site is built. Right now, across 94 results, the median gap between estimate and measurement is 13%, and 78% of estimates land within 25% of the measured number. On average the model runs 14% conservative.

Prices

Prices are typical US street prices when we last checked, with that date shown. For older GPUs we list the used price. Prices change quickly, so treat them as a guide and check before you buy.

Known limits

• Real speed drops as the context fills, because the KV cache also has to be read for every token. Our figures are for short contexts.
• Power limits, thermals, drivers, and runtime settings can move results by 10–30%.
• Very new GPUs often get faster over the first months as runtimes add support.

Found a mistake?

Tell us on X. We fix errors quickly and update the date on the affected pages.

Frequently asked questions

How accurate are the speed estimates?

Across 94 measured single-GPU llama.cpp-class results, the median difference between our estimate and the measurement is 13%, and 78% of estimates are within 25%. Estimates are less reliable for mixture-of-experts models, multi-GPU setups, and very new hardware whose software is still maturing.