A practical, vendor-neutral guide to the computers that run large language models privately, on your own desk — compared on the specs that decide the outcome: how much memory the GPU can reach, how fast that memory is, and how mature the local software stack is.
Buying a local AI machine in 2026 is mostly a memory decision, not a GPU-name contest. Quantization has made model capacity the gating spec and memory bandwidth the speed limit; software maturity is what separates "it runs" from "it fights you." Below: the four things that decide a good purchase, then seven machines worth buying and the five models to test on them.
The first number that matters is GPU-addressable memory — VRAM on a discrete card, or unified memory on Apple, GB10, and AMD Strix Halo systems. It sets the hard ceiling on the model you can load. "AI TOPS" on the box rarely does.
Once a model fits, memory bandwidth sets tokens per second. A 128GB box at 256 GB/s holds a 70B model but generates slowly; a 96GB GPU at 1.7 TB/s is far faster. Capacity gets you in the door; bandwidth decides if you stay.
A global DRAM/LPDDR5x crunch reshaped this market in months: Apple dropped the 512GB Mac Studio, NVIDIA raised DGX Spark to $4,699, and Framework is repricing its 128GB configs. Every price here is a moving target.
For most buyers the win is not the largest model — it is a machine that runs the best 24B–35B-class models well, privately, without cloud bills. Bigger boxes widen headroom; they do not change what most people should run first.
Buy for the models you want to run every day, in private. A machine's memory tier and software stack decide that far more than its parameter-count headline or its price sticker.
Stop comparing GPU model names like a gamer. Think like a local-AI operator and compare these four, in order. The reason capacity comes first is simple: 4-bit and 8-bit quantization cut a model's memory footprint enough that whether it fits — not how fast the chip is — decides what you can run. For the deeper "why bandwidth is the wall" argument, see our KV cache and the memory wall breakdown.
Whether VRAM or unified memory, this is the gate. Quantization (4-bit and 8-bit) shrinks models enough that capacity, not compute, is what lets a machine load a 32B, 70B, or 120B model at all.
After a model fits, bandwidth caps decode speed. Unified-memory boxes land around 256–820 GB/s; a discrete workstation GPU clears 1.5 TB/s. This is usually the real inference bottleneck — not the NPU.
MLX on Apple, CUDA + NIM on NVIDIA, ROCm on AMD. The stack decides what runs without a fight. LM Studio and Ollama cover all three platforms; CUDA is still the broadest path.
Can you grow past one box? DGX Spark and ASUS GX10 link two units for larger models; everything else is distributed serving, not pooled memory. This matters only after the first three.
Prices here are unusually unstable. The 2026 DRAM and LPDDR5x shortage is still working through the market — it is why the 512GB Mac Studio is gone and why DGX Spark now costs $700 more than at launch. The spec that makes these machines special is the same one in short supply, so re-check current pricing and available configs before you commit.
Most readers fall into one of four buckets. Find yours, then read that machine's full card below before deciding.
Most people who want a quiet, capable local-AI desktop
Mac Studio
Spans $1,999 to $3,999+, runs near-silent, has the best single-box bandwidth here (819 GB/s on M3 Ultra), and a mature MLX + LM Studio stack.
Watch out: The 512GB option is gone — 256GB is the current ceiling.
Developers who want an NVIDIA-native appliance
DGX Spark or ASUS Ascent GX10
Turnkey CUDA, NIM, and DGX OS, plus an official two-box link to ~405B-class models. The GX10 is the cheaper route to the same GB10 chip.
Watch out: 273 GB/s bandwidth makes dense 70B models slow; MoE models are the point.
Value buyers who want the most memory per dollar
Framework Desktop, HP Z2 Mini G1a, or GMKtec EVO-X2
x86 unified-memory boxes give you 128GB for a fraction of a 128GB Mac. Framework for repairability, HP for business support, GMKtec for the lowest price.
Watch out: All sit near 256 GB/s — great for MoE, sluggish on dense 70B.
Businesses that think like workstation owners
HP Z2 Tower G1i
Standard CUDA workstation with up to a 96GB RTX PRO 6000 Blackwell card at ~1.7 TB/s — the fastest dense-model inference here, and internal GPU upgrades later.
Watch out: The 96GB GPU config runs past $14,000; capability equals the card you buy.
Complete machines you can buy and run today, not component shopping lists. Each card leads with the spec that matters for local inference. Illustrations are stylized; real product photos can be dropped in later.
Runs well: M4 Max (36–128GB): 7B–32B at 4–8-bit, and 70B at 4-bit with 64GB+. M3 Ultra (96–256GB at 819 GB/s): 70B comfortably, plus 120B–235B-class MoE models at usable speeds.
Struggles with: The 600B-in-memory headline is gone with the 512GB config. Raw training and prompt-processing throughput still favor NVIDIA GPUs.
Strengths
Trade-offs
The default pick for most readers who want a quiet, capable local-AI desktop and live in the Apple-silicon software stack.
Runs well: 20B–120B MoE models at FP4/INT4 (gpt-oss-120B runs ~38–50 tok/s), plus 7B–32B dense. Two linked units (256GB) reach ~405B-class MoE inference.
Struggles with: A dense 70B fits but is bandwidth-bound to roughly 3 tok/s — runnable, not interactive. It is not the tokens-per-dollar leader; a discrete RTX card beats it on raw decode.
Strengths
Trade-offs
The cleanest NVIDIA-native box for CUDA/NIM-first developers, local agents, and "replace the cloud demo" workflows.
Runs well: The same class as DGX Spark — 70B at 4-bit, 120B-class MoE, and a two-unit stack to ~405B over ConnectX-7. Runs DGX OS / Ubuntu.
Struggles with: Identical 273 GB/s ceiling, so dense-model speed is the same story. Channel pricing is inconsistent — there is no single fixed MSRP.
Strengths
Trade-offs
Buyers who want DGX Spark-class GB10 capability at a lower entry price and are comfortable on Ubuntu/DGX OS.
Runs well: 64GB: 20–32B dense models comfortably at Q4–Q6. 128GB: 70B–120B dense and large MoE in unified memory — gpt-oss-120B around 40–48 tok/s.
Struggles with: Dense models near the bandwidth ceiling feel sluggish; MoE is where it shines. The 128GB price keeps climbing with the DRAM shortage.
Strengths
Trade-offs
Enthusiasts who want a conventional, serviceable desktop and the most local headroom per dollar.
Runs well: 100–120B MoE at Int4 (gpt-oss-120B around 24–40 tok/s) using up to 96GB of unified memory as VRAM, plus 7B–32B dense comfortably.
Struggles with: Dense 70B is bandwidth-bound (~4–6 tok/s). The top config is priced as business hardware, not a hobbyist deal.
Strengths
Trade-offs
SMBs that need vendor support, manageability, and realistic office or edge deployment — not a tinkerer’s box.
Runs well: 7B–32B dense at good interactive speed, and 100B+ MoE in the 128GB config — Llama 4 Scout (109B/17B-active) runs around 20 tok/s.
Struggles with: Same bandwidth class as the other minis, and thermals run hot under sustained "performance" load. Support is thin versus HP.
Strengths
Trade-offs
Budget-minded solo developers and hobbyists who want maximum unified memory per dollar and do not need vendor support.
Runs well: With the RTX PRO 6000 (96GB VRAM): 70B-class models at 4-bit fully in VRAM with long context, at far higher tokens/sec than any unified-memory box here.
Struggles with: Capability lives entirely in the GPU you buy — the 96GB card alone is roughly $8,000–$9,000+. System-RAM offload is slow (~90 GB/s).
Strengths
Trade-offs
Businesses that want classic expandable CUDA workstations and the fastest dense-model inference, rather than integrated memory.
The four columns that predict how a machine behaves on real local-LLM work. Read memory and bandwidth together — one caps the model size you can load, the other caps how fast it answers.
| Machine | Memory | Bandwidth | Price snapshot | Practical model tier |
|---|---|---|---|---|
| Mac Studio (M3 Ultra) | Up to 256GB unified | 819 GB/s | From $3,999 | 70B–235B class |
| Mac Studio (M4 Max) | Up to 128GB unified | 410–546 GB/s | From $1,999 | 24B–70B class |
| NVIDIA DGX Spark | 128GB unified | 273 GB/s | $4,699 | 20B–120B MoE |
| ASUS Ascent GX10 | 128GB unified | 273 GB/s | ~$2,999–$3,999 | 20B–120B MoE |
| Framework Desktop | Up to 128GB unified | 256 GB/s | From $1,269 | 24B–120B MoE |
| HP Z2 Mini G1a | Up to 128GB unified | ~256 GB/s | ~$3,300 (128GB) | 24B–120B MoE |
| GMKtec EVO-X2 | Up to 128GB unified | 256 GB/s | ~$1,999 (128GB) | 24B–110B MoE |
| HP Z2 Tower G1i | Up to 96GB VRAM | ~1.7 TB/s (GPU) | >$14,000 (96GB) | 70B at top speed |
A unified-memory box and a workstation GPU can list the same "70B" tier and behave nothing alike. The HP tower's 96GB card at ~1.7 TB/s generates a dense 70B far faster than a 256 GB/s mini that technically also fits it. If interactive speed on big dense models matters, pay for bandwidth.
A good buying guide gives you a test plan, not just a parts list. These five anchor the real use cases — general assistant, multimodal, efficient MoE, reasoning, and a high-end stretch target — and let you judge any machine on the same lens. They pair naturally with our open-source & open-weight models guide.
A practical, current generalist — small enough to matter on local hardware, strong enough for real work, and permissively licensed.
24B dense · 128k context · text + vision · Apache 2.0 · ~13–15GB at Q4
Fit: Runs on any entry box (32–64GB) and up; the everyday workhorse.
The reference for image-plus-text work locally, with broad language coverage.
27B dense · 128k context · text + image · 140+ languages · Gemma license · ~16–17GB at Q4
Fit: Comfortable on a 24GB GPU or any 32–64GB unified box.
One of the best "modern but efficient" MoE targets for coding and agent work — sparsity keeps it fast for its size.
30.5B total / 3.3B active MoE · 32k native (131k via YaRN) · Apache 2.0 · ~17–20GB at Q4
Fit: Entry boxes run it quickly thanks to MoE sparsity; scales up cleanly.
Local chain-of-thought-style reasoning without jumping to a giant model — and an MIT license.
32B dense · 128k context · MIT · ~20GB at Q4
Fit: A 24GB GPU or 32GB+ unified box; the local reasoning benchmark.
The model that separates "good" local boxes from "serious" ones — a large MoE that needs real memory.
109B total / 17B active MoE (16 experts) · 10M-token context (trained at 256k) · Llama 4 license · ~55–60GB at Int4
Fit: Needs a 128GB unified box or a 96GB GPU. Not a consumer-GPU model.
| Model | Approx. memory | 32–64GB box | 128GB box | 256GB / dual GB10 |
|---|---|---|---|---|
| Mistral Small 3.2 24B | ~14GB (Q4) | Easy | Easy | Easy |
| Gemma 3 27B IT | ~17GB (Q4) | Comfortable | Easy | Easy |
| Qwen3-30B-A3B | ~18GB (Q4) | Comfortable (MoE-fast) | Easy | Easy |
| DeepSeek-R1-Distill-Qwen-32B | ~20GB (Q4) | Comfortable | Easy | Easy |
| Llama 4 Scout (109B MoE) | ~55–60GB (Int4) | No | Yes | Easy |
If you have seen "Qwen3.6-35B-A3B" floating around buying guides, it does not exist — the name is an AI-generated artifact that fuses two real Qwen models. The genuine model in that class is Qwen3-30B-A3B (30.5B total / 3.3B active). We verified every model on this page against its primary source.
Yes — but the expansion stories are not equal, and the difference is worth real money. The line that matters is between officially linked operation and generic networked serving.
DGX Spark and ASUS Ascent GX10 each carry ConnectX-7 networking (2× QSFP, 200Gbps). NVIDIA documents linking two units to run models up to ~405B parameters — a real, first-party scale-out path, not a LAN trick.
Any of these boxes can form a small serving cluster. vLLM does multi-node inference over Ray, Ray Serve handles production serving, and LM Studio’s LM Link (end-to-end encrypted via Tailscale) lets one machine use models loaded on another as if they were local.
Two ordinary desktops on Ethernet do not become one big-memory machine. Apple markets no Mac-to-Mac memory pooling; multi-Mac sharding exists only through third-party tools (the open-source exo framework, Thunderbolt 5 RDMA in macOS 26.2). Treat it as advanced DIY.
Mac Studios and AMD minis scale beautifully as an office fleet — separate endpoints, task routing, a model per node. They do not become one pooled-memory supercomputer by plugging into Ethernet. If you need one machine to act bigger, that is a GB10 (DGX Spark / GX10) feature, not a LAN feature.
Hardware gets you the memory; the stack decides whether models actually run. Three ecosystems, plus the cross-platform tools that smooth all of them.
Best for: MLX-first local inference on Apple silicon
MLX is built for unified memory; LM Studio runs both llama.cpp and MLX per model.
Strongest out-of-the-box experience on Mac Studio.
Best for: CUDA-first developer and enterprise workflows
CUDA, NIM microservices, and DGX OS ship on DGX Spark and GX10; RTX towers use the standard CUDA stack.
The broadest ecosystem and the only first-party two-box link.
Best for: x86 unified-memory value on Linux
Ollama has a dedicated ROCm package path on Linux; llama.cpp and Vulkan also work on Strix Halo minis.
Improving fast, but still the least plug-and-play of the three.
LM Studio (Mac, Windows, Linux; llama.cpp and MLX) and Ollama (all three, with a ROCm path on Linux) run everywhere and are the fastest way to start. LM Studio's LM Link even lets a back-room machine serve every desk, end-to-end encrypted over Tailscale — a genuine win for small teams sharing one strong box.
GPU-addressable memory — VRAM on a discrete card, or unified memory on Apple, GB10, and AMD systems. It sets the hard ceiling on the model you can load. Memory bandwidth is the close second, because it decides how fast that model generates text.
A 32–64GB box is the real entry point: it runs the best 24B–35B-class models (Mistral Small 3.2, Gemma 3 27B, Qwen3-30B-A3B, DeepSeek-R1-Distill-Qwen-32B) under practical quantization. 96–128GB opens up 70B dense and large MoE models. 256GB (Mac Studio M3 Ultra) reaches 235B-class work.
If it fits in memory, yes — but bandwidth sets the speed. A dense 70B on a 256 GB/s unified-memory box is runnable but slow (single-digit tokens/sec). The same model on a 96GB workstation GPU at ~1.7 TB/s is far faster. MoE models like gpt-oss-120B are much friendlier to the unified-memory boxes.
Apple for the smoothest MLX-first experience and the best single-box bandwidth. NVIDIA for CUDA-first development, the NIM/DGX stack, and two-box scaling. AMD (Strix Halo minis) for the most unified memory per dollar, if you are comfortable on Linux and ROCm.
DGX Spark and ASUS GX10 officially link two units over ConnectX-7 networking for models up to ~405B. Everything else is distributed serving (vLLM + Ray, LM Studio LM Link), which shares or splits workloads but does not pool memory into one machine.
A global DRAM and LPDDR5x supply crunch. It pushed Apple to drop the 512GB Mac Studio (256GB is now the ceiling), NVIDIA to raise DGX Spark from $3,999 to $4,699, and Framework to reprice its 128GB configs. Memory-heavy machines are the most exposed, so treat every price as a snapshot.
For LLM token generation, mostly no. The NPU is used for on-device features and Core ML-style work, not llama.cpp or MLX decoding, which runs on the GPU. Compare GPU-addressable memory and bandwidth, not the TOPS sticker.
It depends on volume, but cost is rarely the only reason. The real drivers are privacy (data never leaves the building), predictable all-day use with no per-token bills, and offline reliability. For steady, private workloads, a one-time machine purchase often pays back quickly.
Every price and spec on this page was checked against primary sources — manufacturer spec pages, official model cards, and tooling documentation — on June 23, 2026, with model existence adversarially verified against its source repository.