llmrun

Hardware11 min read

RTX 5090 for local LLMs in 2026: what 32 GB buys

While the RTX 5090 has an extra 8 GB of headroom, the best 27B-class models remain the same.

The RTX 5090, the first GeForce with 32 GB of memory, was released by Nvidia in January 2025 for a retail price of $1,999. The new 5090 sold for a median $5,502 in our October 2026 price data, and by mid-October it was gone from Nvidia's own US store. At an estimated text generation speed about 1.8 times as fast as an RTX 4090, the 5090 is still the fastest single desktop card for models people run at home. While higher precision or a longer context is possible with the extra 8 GB, a bigger class of model than the 27B is not. If you already have one, keep it. If you are shopping, you can get most of the capabilities at a fraction of the price with a 24 GB card.

Key takeaways

  1. The main thing is that the 5090 is faster. For Qwen3.8 27B the generation estimates on the 5090 and 4090 are 66.9 tok/s and 37.6 tok/s respectively. The gap in public measurements is closer to 1.5 times.
  2. The best models available for 32 GB are the same as for 24 GB. With the extra memory you get an 8-bit quant or the full context window, and for 70B models you still need 48 GB or more.
  3. Most of the price is a shortage premium. The median new 5090 cost 2.75 times as much in October 2026 as at launch. Price-wise, of course, the 5090 at the November street price costs about twice as much per token as a used RTX 3090.
  4. The card draws 575 W from one 600 W connector. One owner capped it at 400 W and lost about 5% of generation speed, and we would use that setting.

The first GeForce with 32 GB

Jensen Huang made the announcement at CES on 6 January 2025, and Nvidia's launch post called the card "the ultimate GeForce GPU, with more hardware and power than anything we’ve made previously". The Founders Edition came as a surprise. The two-slot card measures 304 mm in length and houses a 575 W chip, cooled by Nvidia's "double flow through" design.

For running models, the memory in Nvidia's spec sheet matters more, with 32 GB of GDDR7 and a bandwidth of 1,792 GB/s. Text generation needs to read every weight for every token, so the speed limit is memory bandwidth, of which the 5090 has about 78% more than a 4090. The RTX 3090 arrived in 2020, and the GeForce memory limit had remained at 24 GB ever since.

The first GeForce with 32 GB, in a two-slot Founders Edition

The launch footnote was odd. In February 2025 TechPowerUp found retail 5090s with 168 render units instead of 176. Nvidia confirmed the issue, said fewer than 0.5% of cards were affected and offered replacements: "Affected consumers can contact the board manufacturer for a replacement." The statement said there was "no impact on AI and Compute workloads", leaving model use unaffected.

One connector, 575 watts

The power connector was a problem. The card draws 575 W through one 12V-2x6 plug specified by Nvidia for a 600 W cable, leaving little margin for a bad contact. In February 2025, less than two weeks after launch, overclocker der8auer measured the current under load through each of the six 12 V wires on his own Founders Edition. PC Gamer reported that one wire carried 22 A, roughly half the card's load, and that within four minutes the "PSU side socket was over 140 °C". All six wires connect to a single rail on the Founders Edition board, so an overloaded wire cannot be detected. "For this kind of power draw, one connector is simply not enough," der8auer concluded.

Falcon Northwest was unable to replicate the result. The PC builder said on X that it had "tested many 5090 Founder's builds … undergoing days of … burn-in", but was "not seeing anything like his setup's results", as Tom's Hardware reported. The reports continued. In September 2026 Tom's Hardware reported that an owner's connectors melted at both ends of his power supply's own cable, despite stock settings and a reduced power limit. PNY refused the warranty claim over the unapproved cable, saying its own approved cable was "the only configuration covered under warranty".

Low power use is cheap insurance for local models. As the August 2026 test report puts it, "Local LLM single-stream generation is memory-bandwidth-bound", so the chip rarely needs full power. Even at a power limit of 400 W, Gemma 4 31B in llama.cpp kept 94.7% of its full-power generation speed, while average draw fell by about 100 W in one owner's 5090 test in August 2026. The stock-limit peak in the same test was 686 W, above the connector's rating.

What the extra 8 GB buys

Does 32 GB give you a better model? By our Score, the answer is no. The top four models are the same on the 5090 and a 24 GB card, with Qwen3.8 27B first and all four fitting in 24 GB at Q4_K_M.

FitModel picks table: The best models on a 5090 are the ones that already fit 24 GBRanked by llmrun Score · best quant that fits · speeds estimated on the RTX 5090
#ModelSpeed on RTX 5090llmrun ScoreQuant · VRAM
1
Q4_K_M · 17.9 GB · 66.9 t/s
66.9 tok/sQ4_K_M · 17.9 GB
2
Q4_K_M · 17.9 GB · 66.9 t/s
66.9 tok/sQ4_K_M · 17.9 GB
3
Q4_K_M · 21.7 GB · 57.1 t/s
57.1 tok/sQ4_K_M · 21.7 GB
4
Q4_K_M · 16.5 GB · 250 t/s
250 tok/sQ4_K_M · 16.5 GB
Source: llmrun catalogue and llmrun Score · Data as of Nov 21, 2026All 349 models on the RTX 5090 →

What changes is how much room each model gets. At Q4_K_M with a 16K context, Qwen3.8 27B needs 17.9 GB, which is a bit over half the card. The whole 262,144-token window at that quant uses 31.3 GB on 32 GB, with the alternative of an 8-bit Q8_0 quant at a 64K context. The limit on a 24 GB card is Q5_K_M for the same context. Our memory breakdown covers all the quants at all the sizes. That headroom makes a real difference for an agent that keeps a whole codebase in context. For chat, it is rarely needed.

The bigger model class is still out of reach. For Llama 3.3 70B, Q4_K_M needs 51.3 GB, while the best quant for 32 GB is the 2-bit IQ2_M, which gets our grade D. For 70B-class models you need 48 GB or more, from two used RTX 3090s, a Mac with enough unified memory or the RTX PRO 6000 Blackwell, with 96 GB on the same chip. See the RTX 5090 page for all 2,692 models that fit.

There are other 32 GB cards available. List prices are $1,299 and $949 for AMD's Radeon AI PRO R9700 and Intel's Arc Pro B70 respectively. Both cards have about a third of the 5090's bandwidth. For the same 27B model in our AMD and Intel guide, the generation estimates are 22.0 and 17.5 tok/s respectively.

How fast it feels

This is where the 5090 earns its reputation. We estimate the generation speed on Qwen3.8 27B at about 67 tokens a second. With adults reading non-fiction at about 238 words a minute, roughly four words a second, even a 4090 already writes faster than you can read. The 5090's lead really comes into play when nobody is reading, for agents calling the model dozens of times per task or batch jobs summarising a folder of documents.

SpeedComparison chart: By our estimate the 5090 is about 1.8× a 4090 on dense models, 1.4× on the MoEtok/s, estimated, Q4_K_M
  • RTX 5090
  • RTX 4090
  • Qwen3.5 9B

    RTX 5090183.1
    RTX 4090103
  • Qwen3 14B

    RTX 5090122.6
    RTX 409069
  • Qwen3.8 27B

    RTX 509066.9
    RTX 409037.6
  • Qwen3.6 35B A3B (MoE)

    RTX 5090213.3
    RTX 4090151
Source: llmrun estimates, /gpu/rtx-5090 and /gpu/rtx-4090 · Data as of Nov 16, 2026

Our dense-model estimate follows the bandwidth almost exactly, while measurements with real cards give a smaller gap. The Qwen3 32B speeds for a 5090 and a 4090 in Hardware Corner's benchmarks, updated in March 2026, were 61.4 tok/s and 39.6 tok/s, respectively, about 1.55 times. The same 1.56 times appears on a small 7B model in llama.cpp's public CUDA scoreboard. By our estimate, generation should reflect most (but not all) of the bandwidth advantage, at one and a half to one and three-quarter times a 4090's speed.

The difference in waiting for the first word is smaller. Reading the whole prompt before answering depends more on compute and less on memory. On the 27B with a 4,000-token prompt, our time-to-first-word estimates are about 2.4, 3.1 and 7.1 seconds for the 5090, 4090 and 3090 respectively. Hardware Corner measured about 3,000 tok/s for prompt reading on a similar Qwen 27B, which would bring the wait closer to a second and a half and put our estimate on the cautious side. Either way, the 5090 and 4090 are close, while the 3090 makes you wait. The speeds given are estimated by llmrun's calibrated model, not measured (How we estimate).

The price went up, then the card went away

That $1,999 launch price did not last long, and it has not come back. The monthly median price of a new 5090 in our data rose to $5,502 in October from $3,757 at the start of 2026. The median went up from $4,461 to $5,376 between August and September, the steepest step. Used 5090s went the same way and sold for a September median of $5,652, more than a new one that month.

PricePrice history chart: The median new RTX 5090 rose 46% between January and October 2026USD, monthly median · dashed line: launch price $1,999
New ≈$5,502
$0$2k$4k$6kFebAprJunAugOctMSRPNew ≈$3,757New ≈$3,627New ≈$3,705New ≈$4,102New ≈$4,212New ≈$4,407New ≈$4,372New ≈$4,461New ≈$5,376New ≈$5,502
New, Oct 2026
$5,502
New, Jan 2026
$3,757
Launch price (MSRP)
$1,999
Source: llmrun price tracking · Data as of Oct 2026RTX 5090: prices and specs →

Then the supply dried up. In mid-October the card was "Not Carried" at Micro Center and removed from Nvidia's US marketplace, as Tom's Hardware reported, while the product page still said "starting at $1,999". Nvidia was rumoured to have moved the chip to its workstation cards. Nvidia's Jeff Fisher denied that the 5090 was discontinued, as Tom's Hardware reported, but said "The supply chain (capacity) is indeed somewhat constrained". Tom's Hardware reported third-party asking prices that same week: "At Amazon, RTX 5090s are selling for about $8,500, but from third-party sellers." That is about our catalogue's launch price for the RTX PRO 6000 Blackwell, with three times the memory, a card whose used median in our data has since risen to $17,260.

On 14 November we last observed a new card at about $6,250, with low confidence in that single listing. Any 5090 price, unless it comes with a date and a source, means little, including ours.

Per token, and renting instead

The price makes more of a difference than the speed. We take the card's price and three years of electricity and spread the cost over what it would generate on Qwen3.8 27B at four hours a day. The cost per token at its October median is about the same as for a used 4090, with speed paying for the 5090's price, and about 1.8 times as much as for a used 3090. At the 14 November listing price of $6,250, it would cost twice as much per token as the 3090.

SpeedHardware comparison table: Per token on Qwen3.8 27B, the 5090 matches a used 4090 and costs 1.8 times a used 3090Best value per column in bold · prices differ in basis (street, used or MSRP), see each row · speeds estimated on Qwen3.8 27B · cost at 4 h/day for 3 years, $0.15/kWh
Source: llmrun catalogue and price tracking · Data as of Nov 21, 2026

Electricity is not the main cost for this sort of use. At $0.15 per kWh, four hours a day at the full 575 W adds up to about $126 of power a year. The shortage accounts for the large part: at $1,999, its launch price, the 5090 would have the lowest cost per token in this table.

There is also the cost of renting to consider. Across 27 offers on 16 November, a 5090 on Vast.ai lists at about $0.46 an hour. At that rate a $6,250 card pays for itself after about 13,500 hours, more than nine years at four hours a day. Check the cloud page before deciding, since the rental rate changes as well. For occasional long jobs, renting is the better option. If you run models every day, need privacy, or work offline, owning still makes sense, and a used RTX 3090 covers most of what the 5090 does for about a quarter of the money.

Verdict

Who the 5090 suits

It does not move you up a model class, and at nearly three times the launch price the only thing you are really paying for is speed. It makes sense near $1,999 for running models all day, with a 24 GB card otherwise the better spend.

Good for

  • If you already own one, keep it and cap it at 400 W
  • Agents and batch jobs running all day, when an estimated 1.5 to 1.8 times a 4090's speed saves real time
  • One card for long contexts or 8-bit quants

Not for

  • 70B models, which need 48 GB or more
  • Chat on 27B models, where a 24 GB card already writes faster than you read
  • Paying today's street price for speed alone, at 1.8 to 2 times a used 3090 per token

Questions

Can an RTX 5090 run a 70B model?
Not well. The 2-bit IQ2_M (30.9 GB) is the largest Llama 3.3 70B quant that fits in 32 GB, and it loses enough quality for our grade D. Offloading layers to system RAM lets a 4-bit 70B run, but much more slowly.
How much faster is the RTX 5090 than the RTX 4090 for LLMs?
About 1.8 times on dense models, by our estimate, and 1.4 times on the Qwen3.6 35B A3B mixture-of-experts model.
Is the RTX 5090's power connector safe for long LLM runs?
Most cards run without trouble, though reports of melted connectors include a power-limited card with a native cable. For long runs, cap the card at 400 W, fully seat the plug and avoid bending the cable near it.
Why do RTX 5090 prices vary so much?
Prices vary widely depending on who is selling and when. Nvidia's list price is still $1,999, while third-party sellers on Amazon asked about $8,500 in mid-October.
Should I rent an RTX 5090 instead of buying one?
For occasional long jobs, yes, renting is the better option. If you run models every day, need them offline or want the data on your own machine, owning makes sense.