Can your computer run a local AI model?

The usual local AI question is which computer to buy. The cheaper question comes first: the machine already on your desk runs some size of model, and which size is set by how much memory is left after the system takes its share.

The short answer

Usable memory is roughly total memory minus what macOS or Windows and your open apps are already holding. A model file at about 4-bit, plus room for the conversation, has to fit inside what is left. Apple Silicon subtracts twice: in llama.cpp’s discussion of the Metal memory split, developers report that macOS withholds a third of unified memory from the GPU below 32GB and a quarter above it, which leaves a 16GB Mac about 10.7GB to load a model into.

• A 16GB Apple Silicon Mac runs an 8B model at 4-bit comfortably. A 14B clears its minimum but not the figure canirun.ai calls recommended.
• An 8GB Nvidia card runs 3B models comfortably and an 8B at 4-bit with little room left for context.
• A 24GB Nvidia card reaches the 27B class at 4-bit, the top of the table below.

None of that needs a purchase to establish, and the checks in the next section take about two minutes.

Find your number

On a Mac. The Apple menu’s About This Mac panel shows the memory the machine shipped with. Apple Silicon memory is not upgradable, so that number is fixed for the life of the computer. Apple’s specification pages list what each model was sold with: the 13-inch M3 MacBook Air, for instance, shipped with 8GB or 16GB and could be configured to 24GB, at 100GB/s of memory bandwidth.

On Windows. Settings, then System, then About gives installed RAM. For the graphics card, Task Manager’s Performance tab shows Dedicated GPU memory and how much of it is in use at that moment. On a PC the card’s memory is the number that matters, not the system RAM, and the in-use figure is the part to subtract.

The arithmetic. Usable memory is total memory, minus what the system and your open apps hold, minus the Metal reserve on Apple Silicon. Compare that against the model file, not against the parameter count: a 4-bit file for an 8B model is about 4.5GB, and the conversation needs room on top of it.

Fitting is not the same as running fast. Generation speed tracks memory bandwidth, and it falls as context fills. We have not repeated that derivation here; the bandwidth arithmetic and the context-depth collapse are on our Mac speeds page, including a measured run where an 11,000-token prompt cut generation to a quarter of the benchmark figure.

Model memory figures below are canirun.ai estimates at roughly 4-bit, checked Sept. 16, 2026, and are not our measurements: Llama 3.2 3B, 1.7GB minimum and 2.8GB recommended; Llama 3.1 8B, 4.5GB and 7.5GB; Qwen3 14B, 7.8GB and 13GB; gpt-oss-20b, 11.7GB and 19.6GB; Qwen 3.5 27B, 15.5GB and 25.9GB.

What your machine runs

This table is keyed on machines people already own rather than machines for sale. The speed column is one small reference model measured across chips, so it describes the machine, not the model you will actually load.

Machine	Usable for a model	Largest class that fits at about 4-bit	Speed on the 7B reference model	Measured or estimate
8GB Apple Silicon laptop, M1, M2 or M3 MacBook Air	about 5.3GB	Llama 3.2 3B comfortably; Llama 3.1 8B clears its 4.5GB minimum but not its 7.5GB recommended	14.19 t/s on M1, 21.70 on M2, 21.34 on M3	Measured at 7B
16GB Apple Silicon laptop, M1-M4 Air or 14-inch Pro	about 10.7GB	Llama 3.1 8B comfortably; Qwen3 14B clears its 7.8GB minimum but not its 13GB recommended	24.11 t/s on M4, 36.41 on M1 Pro	Measured at 7B
24GB Apple Silicon laptop	about 16GB	Qwen3 14B comfortably; gpt-oss-20b clears its 11.7GB minimum	21.34 t/s on M3, 24.11 on M4	Measured at 7B; no measurement exists at 20B in these sources
16GB Mac mini, M4 or used M2 Pro	about 10.7GB	Llama 3.1 8B comfortably; Qwen3 14B above its minimum	24.11 t/s on M4, 37.87 on M2 Pro	Measured at 7B
8GB Nvidia card, RTX 4060 Ti 8GB or RTX 5060	8GB less display use	Llama 3.2 3B comfortably; Llama 3.1 8B above its minimum, below its recommended	63.86 t/s on a 4060 Ti 8GB	Measured at 7B
12GB Nvidia card, RTX 3060 12GB	12GB less display use	Qwen3 14B above its 7.8GB minimum, below its 13GB recommended	75.57 t/s on a 3060 12GB	Measured at 7B
16GB Nvidia card, RTX 5060 Ti 16GB	16GB less display use	gpt-oss-20b above its 11.7GB minimum; Qwen 3.5 27B does not fit, because its 15.5GB minimum leaves a 16GB card almost nothing for context or display use	90.94 t/s on a 5060 Ti 16GB	Measured at 7B
24GB Nvidia card, RTX 3090 or RTX 4090	24GB less display use	Qwen 3.5 27B above its 15.5GB minimum, below its 25.9GB recommended	158.16 t/s on a 3090, 186.21 on a 4090	Measured at 7B

Apple Silicon speeds are the community-maintained llama.cpp Apple Silicon table for LLaMA 7B at Q4_0, as listed there with the chips’ bandwidth: M1 with seven GPU cores at 68GB/s, M2 with eight at 100GB/s, M3 with 10 at 100GB/s, M4 with 10 at 120GB/s, M1 Pro with 16 at 200GB/s and M2 Pro with 16 at 200GB/s. Card speeds are the llama.cpp CUDA table for Llama-2-7B at Q4_0 with full GPU offload: the 4060 Ti 8GB and 5060 Ti 16GB rows on build 89d1029, the 3060 12GB row on baa9255, the 3090 on c76b420 and the 4090 on 2241453. Nvidia lists the RTX 5060 Ti as a 16GB or 8GB card, which is why the memory size, not the card name, belongs in your row.

Three honest limits on that table. Each figure is one contributor’s single run rather than a distribution, and Hard Bargains has not tested any of these machines. The benchmark tables record chips, not memory sizes, so a machine that holds its model entirely in the memory above runs near its chip’s rate and one that spills does not. And a larger model is slower in proportion, because each token reads the weights out of memory again; the arithmetic is on our Mac speeds page rather than repeated here.

Free fixes first

Every fix here costs $0. Each one also costs something that is not money, named beside it.

• Run a lower quantization. canirun.ai’s quantization table puts gpt-oss-20b at about 11.3GB at Q4_K_M and about 7.2GB at Q2_K, which moves it from the edge of a 12GB card to the middle of it. The cost is output quality, and how much quality depends on the model and the task.
• Run a smaller model. Llama 3.2 3B’s 1.7GB minimum fits every machine in the table. The cost is capability: a 3B model is weaker at long reasoning and at code than an 8B.
• Run a mixture-of-experts model. gpt-oss-20b is listed at 21B parameters with 3.6B active per token, so it generates faster than a dense model of similar file size. The cost is that the whole file still has to fit; only the compute per token shrinks.
• Close what is holding the memory. Browser windows and design apps are usually the largest part of the in-use figure. The cost is the tabs.
• Accept partial offload. The community benchmarks above specify full GPU offload precisely because the fallback exists: keep some layers on the GPU and the rest in system memory, and an oversized model still runs. The cost is speed, which drops toward the slower memory’s rate.
• On a Mac, raise the Metal cap. The same llama.cpp discussion documents sudo sysctl iogpu.wired_limit_mb as the way to give the GPU more than the default two-thirds share. The cost is the memory everything else was using, and the setting does not survive a restart unless you make it persistent.

Work through that list before reading the next section. If it ends with a model you can live with, the buying decision is already made.

If you do need to buy

On a Mac, memory is soldered, so the fix is a different Mac rather than an upgrade. Our model-first ladder maps the largest model you want to a memory tier and a machine with dated prices, and local AI memory per dollar prices Macs against Strix Halo boxes and Nvidia cards. Apple’s own store and Apple Certified Refurbished are the direct routes.

On a PC, the smallest fix is usually a card with more memory, not a new computer. Start at our used GPU guide, which prices the card tiers with dated new listings and an eBay sold price, works out dollars per gigabyte and lists the power-supply, slot and length checks each card needs. It also covers new 24GB and 32GB cards from Intel and AMD, which at 32GB cost far less than an RTX 5090 but run llama.cpp through Vulkan, ROCm or SYCL rather than CUDA. Check the card’s length against your case and its power connectors against your supply before ordering; a memory figure is not a compatibility guarantee, and we have not tested any of this hardware.

If you are starting from a budget rather than a machine, the budget ladder runs the other way round — $0, under $500, $800-$1,400 and $2,000-$3,500 — and puts the free fixes above on its first rung.

The two links below are Amazon links. No Amazon price is quoted anywhere on this page, and a manufacturer’s price is not an Amazon offer.

See the ASUS Dual GeForce RTX 5060 Ti 16GB GDDR7 OC Edition on Amazon. This opens an exact product listing, whose title we checked Sept. 16, 2026; its seller, price, condition and stock have not been checked. A 16GB 5060 Ti is the row that measured 90.94 t/s on the reference model above.

Find 16GB graphics cards on Amazon. This opens search results, not a specific product. Results mix the 8GB and 16GB versions of the same card name, so confirm the memory size on the exact listing.

Other stores: Nvidia’s RTX 5060 family page states which cards ship with 16GB, and Newegg, Micro Center and eBay’s used listings are the other places to buy one. At 24GB, a used RTX 3090 comes from a secondhand marketplace; the new 24GB and 32GB alternatives are priced in the GPU guide’s options table.

One last check before spending: buy the tier the model you actually run needs. A machine that clears today’s model by one tier costs considerably less than one that clears every model, and the free fixes above move more rows than most buyers expect.