Processor Gains Now Come From Software, Not Silicon
Article

Processor Gains Now Come From Software, Not Silicon

Across AMD, Nvidia and the memory bottleneck debate, the newest CPU and GPU performance is increasingly unlocked by optimizers rather than by bigger chips.

SuryaNovember 20, 20265 min read

Photo: Tom's Hardware

The defining constraint in processors has shifted from what the silicon can do to what the software lets it do. Two of the freshest data points on this beat - OpenAI's work with AMD and Nvidia's optimization results for Vera - both describe double-digit performance gains achieved without a new chip. A third story, about memory traffic in AI scale-up, explains why squeezing the hardware harder has become the main lever. For US chipmakers, the competitive edge is migrating toward software teams and away from the transistor budget alone.

AMD Finds Headroom With a Partner

The clearest case comes from AMD, where OpenAI helped improve the design of an upcoming flagship GPU by 14% on power, performance and area, as Tom's Hardware reported. The same GPT models also helped Lisa Su's outfit boost its new CPUs. That detail matters more than the headline number. PPA is the currency of chip design, and a 14% improvement is the sort of gain that normally takes a full architectural generation or a process shrink to earn. Here it arrived through model-assisted design work on an existing roadmap item.

The CPU side of that report is the part that belongs squarely on this beat. AMD's processor line has been competing on core counts, cache and efficiency for several generations, and the returns from those levers have become harder to find. If AI models can help identify design and configuration improvements in a new CPU, the pace of iteration changes. It also changes who needs to be at the table: an AI lab becomes a de facto design partner rather than only a customer.

Nvidia Shows the Same Pattern in Reverse

Nvidia's Vera CPU tells the story from the software end. Vera, built on custom Olympus Arm cores, already offers strong performance with unoptimized code, according to Tom's Hardware. At SC26, Nvidia demonstrated that further software optimization improves performance by nearly 39% on certain workloads, and that architectural fine-tuning is needed to fully unlock those cores.

A 39% gain from optimization is not a rounding error. It is larger than many generational jumps, and it required no new silicon. Nvidia's framing is itself revealing: the company is telling customers that the chip they have is not yet the chip they bought. That is a reasonable message for a vendor with a strong software stack, and a harder one for competitors whose differentiation depends on the hardware specification sheet.

The two stories are mirror images. AMD is using AI to improve the design of chips not yet shipped; Nvidia is using software work to improve the output of chips already in customers' hands. Both point to the same conclusion: the performance a US customer experiences is increasingly a function of the tooling layered on top of the processor.

Why Memory Makes This Inevitable

The third story supplies the underlying pressure. More than 80% of scale-up traffic is just memory moving around, according to SiliconANGLE, which describes memory bottlenecks as among the costliest problems in AI infrastructure as inference overtakes training. GPUs sit idle waiting on data, while high-bandwidth memory stays bolted to processors that cannot share it.

That is an indictment of system balance, not of compute. If the processor spends much of its time waiting, then making the processor faster yields less than making the data path better. Optimization work - scheduling, data placement, compiler choices, memory movement - attacks exactly where the idle time lives. Nvidia's near-39% figure and the memory-traffic figure are describing the same phenomenon from different angles. The headroom is in the plumbing.

SiliconANGLE notes that photonics is shaping up as one way out, with Marvell Technology pairing optics with photonic memory. That is a hardware answer to a hardware problem, and it will take time. In the interim, software is the only lever that moves on a quarterly cadence.

The Edge Moves to Software Teams

For US technology companies, the strategic implication is uncomfortable but clear. A processor's competitive position now depends on how much of its potential the vendor's software can extract. Nvidia has built that capability over years and is using it to extend the useful life of its own CPUs. AMD's partnership with OpenAI suggests it is trying to buy or borrow similar capability rather than build it entirely in-house.

That raises the value of compiler engineers, kernel developers and performance-tuning teams relative to raw design headcount. It also raises the cost of falling behind. A vendor that ships competitive silicon with immature tooling can lose the equivalent of a full generation on real customer workloads, which is precisely the gap Nvidia is advertising at SC26.

There is a concentration risk here too. If the best optimization work sits with the largest platform owner, smaller US chipmakers face a widening gap between benchmark specifications and deployed performance. Buyers evaluating processors will need to test on their own workloads rather than trust peak numbers, because the spread between unoptimized and optimized results is now large enough to reverse a purchasing decision.

What This Means for US Buyers

For US consumers and enterprises, the practical consequence is that performance arrives in software updates rather than only with new purchases. Nvidia's demonstration implies that owners of Vera-based systems have meaningful gains still to come from tuning. AMD's design collaboration implies that its next CPUs may land with more competitive margins than the pre-collaboration roadmap implied. Both are good for buyers who already committed to a platform, and both complicate the case for upgrading on a fixed cadence.

The memory bottleneck adds a cost dimension. If most scale-up traffic is data movement, then power and time go into moving bytes rather than doing arithmetic. That is an efficiency problem that shows up in operating costs, and it is why optimization of data movement is worth as much as optimization of compute.

What to Watch

Watch whether AMD discloses how the OpenAI-assisted design work translates into shipping products, and whether the CPU gains are described in comparable terms to the 14% GPU PPA improvement. Watch whether Nvidia's SC26 optimization results appear in general availability or remain workload-specific demonstrations - the company said "certain workloads," and that qualifier matters. Watch whether Marvell's optics-and-photonic-memory approach moves from a proposed path to a product, since that would address the more than 80% memory-traffic figure at the hardware level. And watch the software tooling announcements from US chipmakers generally: on this beat, that is now where performance is won or lost.

More on this beat: Hardware on TechManNews.

#CPUs#Nvidia Vera#AMD#software optimization#memory bandwidth#AI infrastructure

Newsletter

Get Tech News in Your Inbox

The latest AI, gadgets, software and startup stories from TechManNews, delivered every morning - free.