How much memory to run an open model locally

An open-weight model is a file. The question that decides whether you can use it is unglamorous: does the file fit in your machine's memory, all at once? The graphics card, the processor and the benchmark scores all come second. If the weights do not fit, nothing else matters.

That question got harder in 2026. The strongest open models are now very large and very sparse at the same time. A model can hold three quarters of a trillion weights and touch only a fortieth of them to produce each word. It runs far faster than its size suggests, and it still needs every byte of that size in memory.

Total parameters set the bill. Active parameters set the speed.

Almost every frontier open model now uses a mixture of experts. The model is divided into many small specialist networks. For each token, a router picks a few of them. GLM-5.3 holds 744B parameters and uses about 40B per token.¹ DeepSeek-V4.1-Flash holds a 552B backbone and activates 8B parameters while it reads and 16B while it writes.⁶

Think of it as a reference library. You read a few pages to answer a question, so the answer comes quickly. You still need shelf space for every book, because you do not know in advance which pages you will need. The router changes its mind on every token.

This is why a 744B model is usable on a single machine at all, and why that machine still needs hundreds of gigabytes.

What a quantisation really costs

Quantisation stores each weight in fewer bits. It is the main lever you have. GLM-5.3 takes 810GB of memory at 8-bit and 223GB at 1-bit — the same model, a quarter of the footprint.¹

Two things are easy to get wrong. The first is that the file size on disk is not the memory figure. You also hold the context and the working state, so the running total sits above the download. Unsloth's published tables account for this, which is why their memory column reads higher than their file column.

The second is that accuracy falls as the bits come off, and it does not fall gently at the bottom. On Unsloth's own measurements GLM-5.3 keeps 94.29% top-1 accuracy at 4-bit, 80.93% at 2-bit and 72.56% at 1-bit.¹ The 1-bit build is a way to see the model run. It is not the model you would put behind real work.

Three ways to reach the number

One pool of unified memory. On Apple silicon the processor and graphics share the same memory, so a 512GB Mac Studio can give almost all of it to a model. This is the simplest route to a large capacity, and the quietest. Apple's M5 Ultra raises memory bandwidth to 1.2TB/s, about 50% above M3 Ultra, and states plainly that the machine is meant to "run huge LLMs with hundreds of billions of parameters entirely on device".⁷ The 512GB configuration of the new machine is not due until late October.

A graphics card plus system RAM. Mixture-of-experts models split neatly. The attention layers, which every token touches, go on the graphics card. The expert layers, which are large but rarely read, stay in system memory. In llama.cpp the --n-cpu-moe flag controls where the split falls. This is the cheapest way to reach a large capacity, and it moves the bottleneck to the speed of your DDR5, because once the experts live in system memory their read speed is your memory bandwidth. An eight-channel workstation platform matters here far more than the badge on the graphics card.

Everything in VRAM. The fastest and the most expensive. Four NVIDIA RTX PRO 6000 cards give 384GB. HelixML measured GLM-5.3-Flash on four RTX PRO 6000 Server Edition cards at 63.7 output tokens per second for a single request, rising to 82.8 with speculative decoding — and reported that it took a pinned experimental build to get there at all.⁸ Day-zero support for a new architecture is rarely ready.

Context is not free

Every model in the chart above advertises a one-million-token context window. Holding that context costs memory of its own, separate from the weights, and it is the line people forget when they size a machine. How much depends entirely on the architecture. DeepSeek publishes 890 bytes per token for V4.1-Flash, which keeps a full million tokens under a gigabyte.⁶ Older designs are far heavier. Size for the weights, then leave real headroom on top.

The ladder

Read the chart as a shopping list and it resolves into four rungs.

Up to about 32GB runs the good small models. Qwen3.8-27B needs 17GB at 4-bit and has vision and reasoning built in.⁴ A single workstation graphics card covers this, and it is more capable than the number suggests.

Around 256GB is where the frontier starts. DeepSeek-V4-Flash fits losslessly in 169GB, so you run the model exactly as its authors shipped it with no accuracy question at all.² MiniMax M3 fits at 3-bit.³

512GB is the interesting rung in September 2026. It is where GLM-5.3 runs at 4-bit with its accuracy intact, and where DeepSeek-V4.1-Flash will land when something can load it. A Mac Studio M3 Ultra with 512GB reaches it in one box today. A Neox Metal reaches it with 512GB of ECC RDIMM and a card to hold the attention layers.

Past 512GB you are building rather than buying. Kimi K3 needs 610GB at 1-bit and 1.6TB to run losslessly.⁵ That is eight-channel registered ECC memory on a workstation platform, and a conversation about what the model is for.

The specific models will change — three of the six in that chart did not exist in June. The arithmetic will not. Find the total parameter count, pick a quantisation you can defend, add the context, and buy the capacity. For what each current model needs in detail, see our notes on GLM-5.3 and DeepSeek V4.1-Flash.