Which GPU runs your local model: a used RTX 3090, a cheaper 32GB card or nothing

For the October shortlist, use the exact GPU cost worksheet to enter tax, shipping, required extras and any new membership cost. It covers the named shortlist configurations; recheck the exact seller, availability, returns and fit before counting a saving. The older prices below keep their original observation dates.

Based on published specifications, third-party benchmarks and price trackers, not our own testing.

Start with the largest model you will run, not the card. Its memory need decides whether 16GB, 24GB or 32GB is enough, and whether any single card on this page is.

The short answer

The same answers in prose, with each figure's store and date

• Buy nothing if the model’s minimum memory already fits your card with room to spare, or if it is a mixture-of-experts model that runs acceptably with its experts offloaded to system RAM. Check that first.
• 16GB is enough for 7B-20B models. gpt-oss-20b needs 11.7GB at minimum. An RTX 5060 Ti 16GB listed at $790 new at Newegg on Sept. 15. The RTX 5070 Ti cost $360 more there and generated nearly twice as fast in llama.cpp’s 7B test.
• 24GB for 27B-30B models: a used RTX 3090 for speed, a new Arc Pro B60 for price. Qwen3 30B-A3B needs 16.8GB. The 3090 sold for $1,032 on eBay as of Sept. 12, well under its $1,337-$1,463 asks. Intel’s B60 was $649.99 new at Newegg on Sept. 16, but it generated under half as fast in the 7B test and does not run CUDA.
• 32GB no longer means $6,700. On Sept. 16 Newegg listed Intel’s Arc Pro B70 at $1,299.99 and AMD’s Radeon AI Pro R9700 at $1,699.99, both in stock; its lowest RTX 5090 listing was $6,700 on Sept. 15. The 5090 keeps CUDA and generated about 1.5 times as fast as the R9700 in a matched test. A 4-bit 70B, at 39.1GB, fits on none of them.
• For 70B models and gpt-oss-120b, one card falls short. Two used 3090s hold 48GB and measured 16.29 t/s on a 4-bit 70B, with power, slot and case caveats, but cannot hold gpt-oss-120b’s 65.4GB minimum in VRAM. GMKtec’s EVO-X2 with 128GB, $3,499.99 with 1TB at GMKtec’s store on Sept. 16, holds both. Compare the two routes or see which 128GB box to buy.

Exact cards on Amazon, check current prices: ASUS RTX 5060 Ti 16GB, ASRock Arc Pro B60 24GB, ASRock Arc Pro B70 32GB and ASRock Radeon AI Pro R9700 32GB. For a used 3090, search eBay and offer near GetPCParts’ sold price.

Two cards or a 128GB box for 70B and gpt-oss-120b

Supports A dense 70B, and a desktop with room for two cards · gpt-oss-120b, or no desktop to put cards in

If a desktop you already own can take two cards and a 1,000W supply, two used RTX 3090s are the cheaper and faster route to a dense 70B: $2,064 at the Sept. 12 eBay sold price, and 16.29 t/s measured against 4.5 t/s on the 128GB box’s chip, but their 48GB cannot hold gpt-oss-120b’s 65.4GB minimum. GMKtec’s EVO-X2 with 128GB costs more, $3,499.99 with 1TB at GMKtec on Sept. 16, needs no host PC and holds both, and on gpt-oss-120b its chip measured 33.7 t/s against 10-17 t/s for two 3090s with experts pushed into system RAM.

The model you run decides it, and if that model already fits what you own, buy nothing yet.

• Your largest model is a dense 70B, and your desktop has the slots, case room and supply: two used 3090s.
• You run gpt-oss-120b or anything else over 48GB, or you have no desktop to put cards in: the 128GB box.
• You need CUDA or fast prompt processing on long inputs: the cards. The box runs neither CUDA nor long prompts quickly.

Route	Added cost, store, date	Memory	Holds 70B / 120B	Speed 70B	Speed 120B	Power and fit	Next step
Keep your card; run a smaller model, or offload experts to RAM	$0	What you own	Check yours	Depends on your card	Depends on your CPU and RAM	Enough system RAM for the offloaded experts	Buy nothing yet
Two used RTX 3090s, in a PC you own	$2,064 at GetPCParts’ eBay sold price, Sept. 12; $2,674-$2,926 at ResalePrices’ asks, Sept. 15; plus a 1,000W supply if you lack one, not priced	48GB VRAM	Yes / no; runs with experts in system RAM	16.29 t/s, measured (XiongjieDai, May 2024); 14-16 t/s, estimate (Compute Market)	10-17 t/s with experts in RAM, measured (LLM Garage, Jan. 2026)	700W of rated card power; Compute Market recommends 1,000W or more; slot spacing and case room	Two-card caveats; eBay
Two Radeon AI Pro R9700s, in a PC you own	$3,399.98, two at Newegg’s $1,699.99, Sept. 16, our arithmetic	64GB VRAM	Yes / no; 64GB is under 65.4GB	No measured figure found	No measured figure found	600W of rated card power, 300W each; no two-card supply figure found; no CUDA	ASRock R9700 Creator on Amazon, check current price; Newegg
GMKtec EVO-X2, 128GB	$3,499.99 with 1TB, $3,649.99 with 2TB, GMKtec, Sept. 16; a complete machine	128GB shared; up to 96GB to the GPU on Windows, per HP and Framework	Yes / yes	4.5 t/s, measured on a Framework Desktop with the same chip (lhl)	33.7 t/s, measured on the same Framework Desktop (lhl)	GMKtec lists 120W sustained, 140W peak and an adapter of about 230W; no host PC; no CUDA	Which 128GB box; GMKtec’s store; EVO-X2 128GB/2TB on Amazon, check current price

Rows are ordered by added cost. The Framework Desktop uses the same chip at $3,449 before storage, but Framework listed it out of stock Sept. 16.

How far the speeds compare. Each figure is one contributor’s run on its own system. The 16.29 t/s pair figure is Llama 3 70B at Q4_K_M in XiongjieDai’s GPU benchmark repository: the average speed of generating 1,024 tokens in llama.cpp on RunPod systems, published as a May 2024 snapshot, with no llama.cpp build named. The 4.5 and 33.7 t/s figures are llama-bench’s 128-token test in lhl’s runs on a pre-production 128GB Framework Desktop: Shisa V2 70B, a Llama 3 70B fine-tune, at i1-Q4_K_M on the HIP rocWMMA backend, and gpt-oss-120b in MXFP4 on Vulkan AMDVLK. The 70B gap does not rest on those two runs alone. Dividing the chip’s 256GB/s memory bandwidth by the 70B’s 39.1GB gives an arithmetic ceiling of about 6.5 t/s, as the 128GB guide sets out, well under the pair’s measured figure. Prompt processing points the same way: 393.89 t/s for the pair on a 1,024-token prompt in the same repository, against 94.7 t/s for the Framework Desktop at 512 tokens, on different prompt lengths.

The gpt-oss-120b comparison favors the box for a different reason. The model activates a fraction of its parameters per token, so the box’s shared memory holds it outright, while two 3090s must push expert layers into system RAM. LLM Garage’s 10-17 t/s run used 64GB of DDR4, a Ryzen 7 5800X and a 38,000-token context; your CPU and memory will move that figure.

Power and fit. Nvidia rates each 3090 at 350W, so the pair is 700W of rated card power before the rest of the PC. Two R9700s are 600W, and we found no power supply figure for a pair. GMKtec says the EVO-X2 runs steadily at 120W and peaks at 140W, and lists an adapter of about 230W. Those are maker ratings, not wall-power measurements.

Choose a GPU by the model you want to run

Supports A 7B-20B model: 16GB is enough · A 27B-30B model, in a PC you own · 24GB for less, if your software doesn't need CUDA · 32GB on one card · A dense 70B, and a desktop with room for two cards · gpt-oss-120b, or no desktop to put cards in

Find the row for the largest model you will actually run. Minimums are canirun.ai estimates at roughly 4-bit quantization; long prompts need more. Each card line names its software: CUDA for Nvidia, and Vulkan, ROCm or SYCL for the AMD and Intel cards.

Minimums used: Llama 3.1 8B 4.5GB, Qwen3 14B 7.8GB, gpt-oss-20b 11.7GB, Qwen 3.5 27B 15.5GB, Qwen3 30B-A3B 16.8GB, Llama 3.3 70B 39.1GB and gpt-oss-120b 65.4GB. canirun.ai’s recommended figures are higher: 19.6GB for gpt-oss-20b and 27.9GB for Qwen3 30B-A3B.

Compare the options

Every card side by side: memory, two llama.cpp speeds, dated prices, power and fit

Supports A 7B-20B model: 16GB is enough · A 27B-30B model, in a PC you own · 24GB for less, if your software doesn't need CUDA · 32GB on one card · A dense 70B, and a desktop with room for two cards · gpt-oss-120b, or no desktop to put cards in

Option	Memory	7B Q4_0 generation	gpt-oss-20b generation	Price, dated	What else must be true	Where to look
Keep your PC; offload experts to system RAM	What you own	Depends on your card	—	$0	Enough system RAM for the offloaded experts; llama.cpp’s guide sets up gpt-oss-120b for 8GB of VRAM with --n-cpu-moe 35 but gives no speed	canirun.ai minimums
RTX 5060 Ti 16GB, new	16GB; 128-bit memory interface	90.94 t/s	111.51 t/s	$790 lowest new listing at Newegg; used eBay listings from $545 (GPUDojo, Sept. 15)	600W power supply; one 8-pin cable or a PCIe Gen 5 cable rated 300W or more; size varies by board	Where to buy
RTX 5070 Ti, new	16GB; 256-bit	176.85 t/s	189.45 t/s	$1,150 lowest new listing at Newegg; one new-condition eBay listing at $995 (GPUDojo, Sept. 15)	750W; two 8-pin cables through the included adapter or a PCIe Gen 5 cable rated 300W or more; size varies by board	Where to buy
Arc Pro B60, new	24GB; 456GB/s	68.55 t/s (Vulkan)	Not in the guide	$649.99 for ASRock’s B60 Creator, sold by Newegg, in stock (Sept. 16)	No CUDA. Intel lists 200W board power; ASRock’s board takes one 8-pin cable, two slots and 271mm, and ASRock advises 30mm more for the cable	Where to buy
RTX 3090, used	24GB; 384-bit	158.16 t/s	161.77 t/s	$1,032 eBay sold-listing price (GetPCParts, Sept. 12); $1,337-$1,463 fair asking range across 291 eBay listings (ResalePrices, Sept. 15)	750W; the Founders Edition uses two 8-pin cables through a 12-pin adapter, takes three slots and is 313mm long; partner boards differ	Used routes; checks
RX 7900 XTX, used	24GB	182.63 t/s (Vulkan)	Not in the guide	eBay listings from $900; lowest new listing $1,348 at Newegg (GPUDojo, Sept. 16)	No CUDA; on AMD’s ROCm list for Linux. Power, connectors and size vary by board	Used routes; checks
Two used RTX 3090s	48GB total	Not measured here	Not in the guide; Llama 3 70B at Q4_K_M: 16.29 t/s, measured (XiongjieDai, May 2024), against Compute Market’s 14-16 t/s estimate; gpt-oss-120b with 26 expert layers in RAM: 10-17 t/s (LLM Garage)	$2,064 at the sold price; $2,674-$2,926 at asks (our arithmetic)	700W of rated card power; Compute Market recommends at least a 1,000W supply; two slots spaced for the coolers; case room	Two cards or a 128GB box; two-card caveats
Arc Pro B70, new	32GB; 608GB/s	112.02 t/s (Vulkan)	Not in the guide	$1,299.99 for ASRock’s B70 Creator, sold by Newegg, in stock (Sept. 16)	No CUDA. Intel lists 230W board power; Newegg lists ASRock’s board with one 12V-2x6 connector, two slots and 271mm; no power supply figure found	Where to buy
Radeon AI Pro R9700, new	32GB; 640GB/s	145.67 t/s (Vulkan)	Not in the guide	$1,699.99 for ASRock’s R9700 Creator, sold by Newegg, in stock (Sept. 16)	No CUDA; on AMD’s ROCm list for Linux. 300W; Newegg lists a 750W supply, two slots and 271mm for ASRock’s board; XFX’s version uses one 12V-2x6 connector	Where to buy; CUDA or not
RTX 5090, new	32GB; 512-bit	290.02 t/s	282.51 t/s	$6,700 lowest new listing at Newegg (GPUDojo, Sept. 15); $8,699 there at 02:18 UTC Sept. 16; no eBay offer	1,000W; four 8-pin cables through an adapter or one 600W PCIe Gen 5 cable; the Founders Edition takes two slots and is 304mm long	Where to buy
GMKtec EVO-X2 128GB, or another Ryzen AI Max+ 395 box	128GB shared; up to 96GB to the GPU on Windows; 256GB/s	Not in these tables	Not in the guide; on a Framework Desktop with the same chip, 70B: 4.5 t/s and gpt-oss-120b: 33.7 t/s, measured (lhl)	EVO-X2 $3,499.99 with 1TB and $3,649.99 with 2TB at GMKtec, listed as available Sept. 16; Framework Desktop $3,449 before storage, out of stock Sept. 16	No desktop PC or power supply upgrade; no CUDA; GMKtec lists 120W sustained and 140W peak	Which 128GB box; two cards or this

Speeds are llama.cpp’s llama-bench tg128 test, which generates 128 tokens with no prompt, so they are short-context ceilings. The 7B column is Llama 2 7B at Q4_0 with flash attention off. Nvidia rows come from llama.cpp’s CUDA scoreboard: builds c76b420 for the 3090, 89d1029 for the 5060 Ti, 933414c for the 5070 Ti and 8cf6b42 for the 5090. Rows marked Vulkan come from its Vulkan scoreboard, which uses the same model, quantization and test: builds dd1ea52 for the R9700, b863507 for the B70, 516a4ca for the B60 and 304665f for the 7900 XTX. Those rows list no driver or operating system, and each reflects a different backend on a different system. The gpt-oss-20b column has flash attention on, from llama.cpp’s gpt-oss guide: build a094f38, except 9ef6b0b for the 5060 Ti. Each row is one contributor’s single system. The two columns use different models and settings, so compare cards within a column, not across columns. The 70B and gpt-oss-120b figures in the two-card and 128GB rows use other tests, described in the two-card comparison.

Long prompts cost time. In the same gpt-oss guide, the 3090’s prompt processing fell from 5,170.56 t/s at 2,048 tokens to 3,577.10 at 32,768, and the 5060 Ti’s fell from 3,839.21 to 3,078.06. Those rows don’t report generation speed at depth.

Nvidia lists each card’s memory interface width, not its bandwidth. The 5060 Ti’s 128-bit interface is half as wide as the 5070 Ti’s 256-bit one. With the same 16GB, the 5070 Ti generated 176.85 t/s to the 5060 Ti’s 90.94 in the 7B test. That fits the pattern that generation speed follows how fast memory moves, which the Mac speed explainer lays out for Apple chips. Intel lists bandwidth directly: 456GB/s for the B60 and 608GB/s for the B70. XFX lists the R9700 with a 256-bit bus at 20Gbps, which works out to 640GB/s. On the Vulkan scoreboard those three generated 68.55, 112.02 and 145.67 t/s, in the same order.

At these dated prices, VRAM cost about $27 per gigabyte in a new Arc Pro B60, $41 in a new Arc Pro B70, $43 in a used 3090 at its sold price, $49 in a new 5060 Ti 16GB, $53 in a new R9700, $72 in a new 5070 Ti and $209 in a new 5090 at its Sept. 15 listing. A used 7900 XTX at its lowest eBay ask works out to about $38. That is our arithmetic. Our August memory comparison used a reported 5090 street price of about $4,700; the $6,700 figure is a Sept. 15 Newegg listing.

CUDA, Vulkan or ROCm: what the cheaper cards cost you

What llama.cpp, Ollama and vLLM support on AMD and Intel cards, and the speed cost

Supports 24GB for less, if your software doesn't need CUDA · 32GB on one card

At 32GB, the AMD and Intel cards cost a fraction of the RTX 5090 per gigabyte, and at 24GB the new B60 undercuts a used 3090, but none of them runs CUDA, Nvidia’s software layer. Whether that matters depends on the program you will actually run.

• llama.cpp runs all of them. Its README lists CUDA for Nvidia, HIP for AMD, SYCL for Intel and Vulkan for GPUs generally. Every AMD and Intel speed on this page is a Vulkan run.
• Ollama documents ROCm support on Linux for cards including the R9700 and RX 7900 XTX, and says Vulkan provides additional GPU support on Windows and Linux. It has no dedicated Intel section and links to driver instructions instead.
• vLLM publishes install instructions for CUDA, ROCm and Intel XPU. Its ROCm list names the Radeon RX 7900 series and the gfx1201 chip, which AMD’s ROCm compatibility page lists as the R9700’s. Its Intel page covers Arc GPUs generally without naming the Arc Pro models.
• AMD’s ROCm page is for Linux. For ROCm 7.14 it lists the R9700 and RX 7900 XTX as supported on Ubuntu, RHEL, Debian and other distributions.
• Speed is the other cost. In one contributor’s run of Qwen3.5 35B-A3B at Q4_K_XL on a single llama.cpp build, an RTX 5090 on CUDA generated 194.0 t/s to the R9700’s 127.4 on Vulkan. The 5090 processed a 512-token prompt at 7,026 t/s to the R9700’s 2,713, and at 32,768 tokens the gap widened to 6,461 against 1,877. The two systems had different CPUs. On the Vulkan scoreboard, the B60 processed prompts at 522.36 t/s, against 3,379.00 on the B70.

The rule: if the install page for your tool lists only CUDA, price the Nvidia rows. If it lists your AMD or Intel card, or you run llama.cpp, those rows are real options.

Two used 3090s: 48GB, with conditions

Two used 3090s: the measured 70B run, power, slots and PCIe lanes

Supports A dense 70B, and a desktop with room for two cards

Two 3090s hold 48GB, enough for Llama 3.3 70B’s 39.1GB minimum. The one measured generation figure we found for the pair is in XiongjieDai’s GPU benchmark repository: 16.29 t/s on Llama 3 70B at Q4_K_M, the average speed of generating 1,024 tokens in llama.cpp on RunPod systems, published as a May 2024 snapshot. The repository names no llama.cpp build. Compute Market’s multi-GPU guide, updated July 19, estimates about 14-16 t/s for a 70B model at Q4_K_M from community benchmarks. That is an estimate, not a measurement, and it sits just under the measured figure. Two 32GB cards from Intel or AMD would hold 64GB, but we found no measured 70B speed for either pair; Puget Systems’ dual-R9700 tests stop at a 27B model.

The other measured two-card figure is for gpt-oss-120b, which does not fit in 48GB. LLM Garage ran gpt-oss-120b at Q4_K_M on two 3090s in January 2026, with --n-cpu-moe 26 holding expert layers in 64GB of DDR4 system RAM. The system used a Ryzen 7 5800X and a 38,000-token context in llama.cpp’s server, and it reported 10-17 t/s generation.

Nvidia rates each 3090 at 350W of card power and a single-card system at 750W. Two cards add up to 700W of rated card power before the rest of the PC, and Compute Market recommends at least a 1,000W supply for the pair. The motherboard needs two slots far enough apart for both coolers, and the case needs room for both cards; the Founders Edition alone takes three slots. Check each slot’s PCIe lane count in the board manual before buying a second card.

Where to buy

Stores, exact Amazon listings and used routes for every card

Supports A 7B-20B model: 16GB is enough · A 27B-30B model, in a PC you own · 24GB for less, if your software doesn't need CUDA · 32GB on one card · A dense 70B, and a desktop with room for two cards · gpt-oss-120b, or no desktop to put cards in

Start with what you own. Load the model on your current machine first. That costs nothing and shows whether memory or speed is the real limit.

New Nvidia cards. Board partners sell each chip in different lengths, widths and connector layouts. Confirm that the exact board fits your case and power supply before ordering.

Amazon search: RTX 5060 Ti 16GB cards. The 5060 Ti also comes with 8GB, so confirm 16GB on the exact listing.

ASUS Dual RTX 5060 Ti 16GB on Amazon, check current price.

ASUS Prime GeForce RTX 5070 Ti 16GB on Amazon, check current price. Listed as a 2.5-slot card.

MSI GeForce RTX 5090 32G Ventus 3X OC on Amazon, check current price.

Amazon search: 1,000W ATX power supplies. Match the wattage Nvidia lists for your card and the connectors your exact board uses.

GPUDojo’s pages for the RTX 5060 Ti 16GB, RTX 5070 Ti, RTX 5090, RTX 3090 and RX 7900 XTX show current listings by store, and Newegg’s RTX 5060 Ti 16GB results are another starting point.

New Intel and AMD cards. The three exact listings below are the ASRock Creator models priced above at Newegg, which lists each as a two-slot card 271mm long. Check the software section above before choosing one over an Nvidia card.

ASRock Intel Arc Pro B60 Creator 24GB on Amazon, check current price.

ASRock Intel Arc Pro B70 Creator 32GB on Amazon, check current price.

ASRock Radeon AI Pro R9700 Creator 32GB on Amazon, check current price.

Newegg’s own pages for the B60, B70 and R9700 showed those prices on Sept. 16.

Used RTX 3090 or RX 7900 XTX. eBay listings for the 3090 and the 7900 XTX, r/hardwareswap and local pickup, where you can watch the card run before paying. GetPCParts tracks 3090 eBay sold prices, and ResalePrices tracks active asks.

A 128GB machine instead. GMKtec’s store listed the EVO-X2 with 128GB at $3,499.99 with 1TB and $3,649.99 with 2TB on Sept. 16, both as available. GMKtec EVO-X2 128GB/2TB on Amazon, check current price. The Framework Desktop was listed out of stock Sept. 16. Two cards or a 128GB box sets the routes side by side, five Ryzen AI Max+ 395 machines are priced and compared here, and the memory comparison covers DGX Spark and used Mac Studios.

Check a used card before and after paying

Five checks before and after paying for a used card

Supports A 27B-30B model, in a PC you own · A dense 70B, and a desktop with room for two cards

1. Match the exact board, not just the chip name. Length, slot width and power connectors differ by maker. Nvidia’s 3090 Founders Edition is 313mm long, takes three slots and uses two 8-pin cables through a 12-pin adapter.
2. Ask why the card is being sold. Tom’s Hardware suggests asking for a photo or video of the card running a game with the date visible.
3. Read the return terms before paying. Without returns, a remote purchase leaves any defect with you.
4. Test the card as soon as it arrives. Tom’s Hardware recommends looping a demanding benchmark such as 3DMark, or a torture test such as FurMark, while watching temperatures, fan speeds and power use. Then load the model you bought it for at the context length you use.
5. Don’t treat a passing test as a history report. Tom’s Hardware warns that mining runs cards at full power for long stretches, which can shorten their lives. A clean benchmark shows that the card works now, not how it was used.

Buy nothing yet

Supports The model already fits your card

The model already fits. Compare canirun.ai’s minimum with your card’s VRAM, then leave headroom, because the recommended figures are higher. In llama.cpp’s gpt-oss guide, gpt-oss-20b ran entirely on a 16GB RTX 5060 Ti, including prompt processing at 32,768 tokens.

Offloading is fast enough. gpt-oss-120b activates 5.1B of its 117B parameters per token, so llama.cpp can keep attention on the GPU and put expert layers in system RAM. Its gpt-oss guide configures the model for 8GB of VRAM with a 32,768-token context. It gives no speed, which depends on your CPU and memory. Try it on the machine you own; if the speed is acceptable for your work, a new card buys less than its price suggests.

A subscription or a rented card does the job. Open models that fit one card are a tier below the frontier services, and a card used a few hours a month is a card you could have rented instead: RunPod listed a 24GB RTX 3090 at $0.22 an hour and a 48GB RTX A6000 at $0.33 on its community tier Sept. 16. The four-way comparison weighs local capacity against $200-a-month plans and against those hourly rates, and gives the hours a month at which buying starts to cost less.

The used price is off. Asks for the 3090 sat well above its recent sold price in mid-September. A seller’s ask is not a market price, and waiting for a listing near recent sales costs you only time.

Running models on a Mac instead? See measured speeds and memory tiers by chip.