Aug. 26, 2026

How many tokens per second do you actually get on a Mac?

The question, verbatim from Hacker News about the new Macs: "How many tokens per second would you even get on them?" Nobody knows; no M5 or M6 machine has been benchmarked, and they ship Sept. 22. What exists today is measured numbers for the current chips, a scaling rule that predicts them well and some clearly labeled arithmetic for the new ones.

On the machines you can buy today, measured: a 7B model at 4-bit runs about 83 tokens a second on an M4 Max and 92-94 on an M2 or M3 Ultra. gpt-oss-120B runs about 80 on an M2 Ultra. DeepSeek R1 671B at 4-bit runs 16-18 on a 512GB M3 Ultra with an empty context — and 4 to 11 once the context has 11,000 tokens in it. The new M5 and M6 Macs have no benchmarks because no reviewer has one; everything about them below is arithmetic and labeled as such.

Measured Mac token-generation speeds by chip

The community-maintained llama.cpp Apple Silicon benchmark runs one model — LLaMA 7B at Q4_0 — across every chip, which makes it the cleanest chip-to-chip comparison that exists. PP512 is prompt processing; TG128 is token generation. Both measured:

Nobody buys a Mac Studio to run a 7B model. The table’s value is the ratios: generation speed tracks the bandwidth column, not the GPU core count and not the chip generation. An M4 Max from 2024 and an M1 Ultra from 2022 generate at the same speed because 546GB/s and a sublinearly-used 800GB/s land in the same place.

How fast do large LLMs run on a Mac?

For gpt-oss-120B, the llama.cpp gpt-oss benchmarks record an M2 Ultra (192GB, 76-core) at 79.68 t/s generation. Olares published a served, single-stream figure of 69.39 t/s on an M3 Ultra — slower because serving stacks add overhead, and worth keeping because it is closer to what a user of a running system sees.

The biggest model that fits in one Mac is the stress test that counts. DeepSeek R1 671B at 4-bit on a 512GB M3 Ultra generates 16-18 t/s with a near-empty context — the number MacRumors carried when the machine launched. Feed it a real conversation and the story changes: at an 11,000-token prompt, one measured run got 4.26 t/s generation under llama.cpp and 11.15 under MLX, with the llama.cpp job taking about nine minutes end to end for the prompt plus a 1,662-token reply. Context depth, not model size, is where these machines actually hurt.

In between: Qwen3-235B-A22B runs about 16 t/s as GGUF and 24 t/s under MLX on an M3 Ultra, per MacStories’ testing. The runtime gap is general — MLX typically runs 10-30% faster than llama.cpp on the same hardware — so a tokens-per-second claim without a runtime label is missing a third of its meaning. Every figure in this piece names its runtime.

How can you estimate Mac tokens per second?

Token generation is memory-bandwidth-bound: each new token requires reading essentially the whole model from memory. TensorFoundry’s roofline analysis of Apple Silicon puts real-world generation at about 75% of bandwidth divided by model size at 4-bit and about 87% at 8-bit. As a rule of thumb: bandwidth divided by model size in gigabytes, then take about three-quarters.

The rule degrades at the small end. Doubling bandwidth from M1 Max to M1 Ultra moved 7B generation from 61.19 to 83.73 — 1.37x, our arithmetic on the measured table — because small quantized models spend compute dequantizing instead of saturating the memory bus. The llama.cpp maintainer’s framing: quantized kernels “require extra compute to dequantize the data compared to F16 models.” Larger and less-quantized models get closer to the full bandwidth ratio.

Prompt processing is a different machine. It is compute-bound — “at large batch size the computation is compute bound,” per the same maintainer — and an MLX discussion of prefill performance gives the arithmetic: time to first token scales with roughly 2 x parameters x context length over available TFLOPS, and quantization does not help, because prefill reads the weights once and computes against every token. That is how a machine that generates 16 t/s spends two and a half minutes silent before the first token of a reply to an 11,000-token prompt.

This split is why Apple’s launch slides need translation. The claimed 3.9x (M5 Max) and 4x (M5 Ultra) gains are for “LLM prompt processing in LM Studio” — vendor-claimed, prompt-side numbers. They are plausible given the new accelerator hardware and they imply nothing about generation speed. Generation is bounded by the bandwidth gains, which are official and much smaller: 546 to 614GB/s on the Max, up 12.5%, and 819GB/s to 1.2TB/s on the Ultra, up 50%.

Estimated M5 and M6 Mac speeds, not measured

No measurement exists for any machine below. These are ceilings computed from official bandwidth specifications, assuming bandwidth-bound generation at the measured efficiency of each chip’s predecessor. Measured numbers arrive when the machines ship Sept. 22; we will replace this arithmetic with measurement.

The derivations, so they can be checked: the M5 Max 7B figure scales the M4 Max’s measured 83.06 by the bandwidth ratio 614/546. The M5 Ultra rows scale M3 Ultra measurements by 1.5, the ratio of 1.2TB/s to 819GB/s — the gpt-oss row from Olares’ served 69.39, the Qwen3 row from MacStories’ 24 under MLX, the DeepSeek row from the 16-18 empty-context range. Scaling the M2 Ultra’s llama-bench 79.68 instead would put gpt-oss near 120; serving overhead is real, so we quote the lower basis. The 70B row is pure roofline — 614GB/s over a ~40GB model at 0.7-0.9 efficiency gives 11-14 — because measured 70B numbers in public sources are too thin to scale from. The M5 Pro and M6 rows scale same-tier predecessor measurements from the llama.cpp table by their bandwidth ratios. All of these are empty-context ceilings; the DeepSeek context-depth collapse above applies to every row in proportion.

Which Mac should you choose for local AI?

Bandwidth and memory size per dollar across every machine that runs models locally is in the memory table. Whether the model you want fits in one Mac — and what the subscription alternative costs — is in the fit comparison. If it does not fit in one, the cluster math covers what stacking Studios does and does not buy, and generation speed across boxes is worse than any row in this piece.

About the data: Community benchmarks are single-run, build-dependent and span mixed configurations; treat every figure as one observation, not a distribution. The Q4_0 reference quantization in the llama.cpp table differs from the quants people actually run by 10-30% either way. Headline generation figures are measured at near-empty context and fall sharply as context fills, as shown in place. The new-machine ceilings are arithmetic from official bandwidth specifications and predecessor measurements, not measurements.