AI infrastructure requirements evolving beyond raw compute capacity

Pietro Scalia, Head of Power System Architecture and Marketing at Renesas Electronics, explains how AI’s burgeoning power demands, amplified by inference workloads, are triggering a fundamental redesign of infrastructure architectures.

For much of the artificial intelligence boom, the conversation has been dominated by compute. The race to build larger and more powerful AI models created unprecedented demand for GPUs, accelerators and high-performance compute clusters. However, as the industry transitions from a training-centric era toward one increasingly defined by inference, agentic AI and eventually physical AI, the constraints shaping infrastructure design are changing.

According to Scalia, the next phase of AI infrastructure will be determined not simply by how much compute can be deployed, but by how effectively systems can power, connect, manage and cool that compute.

“The training age has now started transitioning to inference and agentic AI,” says Scalia. “It’s a completely different scenario.”

As AI infrastructure evolves, technologies such as digital power delivery, memory interfaces, advanced semiconductor packaging and embedded intelligence are becoming every bit as important as the processors they support.

The result is a fundamental rethink of data centre architecture, creating new opportunities and challenges across the entire power electronics ecosystem.

From training to inference: A different infrastructure problem
The first wave of generative AI deployment was focused primarily on training increasingly sophisticated foundation models. Training workloads demand enormous compute clusters operating at maximum performance for extended periods. As organisations move towards deploying AI at scale, however, inference is rapidly becoming the dominant workload.

Inference presents different performance characteristics and places new demands on infrastructure.

One of the most significant factors is memory. While AI processors continue to grow in size and capability, memory capacity and bandwidth have emerged as critical bottlenecks. Scalia points to the rapid evolution of leading accelerator roadmaps as evidence of how difficult balancing compute and memory requirements has become.

As GPU vendors push toward increasingly powerful chiplet-based architectures, they face growing challenges around integrating memory while simultaneously managing escalating power consumption. Designs originally intended to combine multiple high-power compute dies within a single package have encountered practical limitations related to power delivery, thermal management and memory integration.

The result is a growing recognition that simply adding more compute is no longer sufficient. Instead, future AI platforms will depend on balancing processing capability with efficient memory access and data movement. This becomes particularly important as inference workloads expand and AI systems increasingly need to respond in real time. Memory performance is therefore becoming inseparable from overall system performance.

“The challenge is really driven by the amount of memory,” Scalia explains. “The more we move closer to agentic AI, memory will still be very important.”

This trend extends beyond hyperscale data centres. As AI functionality begins moving closer to end devices and edge environments, memory constraints will follow. Whether supporting inference in cloud-scale facilities or enabling physical AI applications in intelligent systems, efficient memory architectures will remain central to performance.

The new bottleneck: Delivering power at scale
While memory may be one critical constraint, power delivery is emerging as another.

Historically, power was considered a supporting function within computing systems. Today, however, delivering energy efficiently to increasingly power-hungry processors has become a central engineering challenge. Modern AI accelerators already consume extraordinary amounts of power. Future generations are expected to push this further, demanding thousands of amps delivered with microscopic precision and near-instantaneous response times.

“The challenge is when we are talking about 6,000 amps, 8,000 amps, 10,000 amps,” says Scalia.

At these levels, power delivery ceases to be solely an electrical engineering problem. Instead, it becomes an interconnected electromechanical and thermal design challenge. Power semiconductor innovation is therefore shifting beyond individual devices and towards sophisticated packaging approaches that integrate multiple elements into highly compact power delivery systems. Manufacturers are increasingly incorporating capacitors, thermal structures and potentially inductive components directly within advanced power packages.

As current requirements climb, innovations in three-dimensional power architectures are becoming essential.

“It’s not really about the chip itself only,” Scalia notes. “It’s the way the chip can convey the current to the XPU and the way the chip can dissipate heat.”

The consequence is a new wave of research and development centred on packaging, current distribution, thermal performance and integration density.

Why 800-volt architectures are becoming the new standard
One of the most visible examples of this transformation is the industry’s growing adoption of 800-volt DC power distribution architectures. Until recently, 48 V systems represented the dominant approach for modern server power distribution. However, the rapid increase in rack power densities has exposed limitations in lower-voltage architectures. Once rack power exceeds roughly 250 kW to 350 kW, current levels become increasingly difficult to manage efficiently.

Higher-voltage distribution offers a practical alternative. By increasing voltage from 48 V to 800 V, current requirements fall dramatically, reducing losses throughout the distribution network and making high-density compute racks more feasible.

Scalia believes industry consensus around 800V has largely been achieved.

“As soon as the rack goes beyond 250, 300 or 350 kilowatts, it makes high-voltage distribution essential,” he says.

Importantly, the transition benefits from experience accumulated in other sectors, particularly electric vehicles, where 800 V architectures have gained significant traction over recent years.

Within AI infrastructure, the move to 800 V is also accelerating adoption of advanced semiconductor materials and power conversion technologies that offer higher efficiency and power density. The industry conversation has shifted from whether 800 V architectures will arrive to how rapidly deployments can scale.

“It’s not anymore a trend,” Scalia says. “It’s more a reality.”

The rise of gallium nitride
As power densities continue increasing, semiconductor materials are becoming a strategic differentiator.

Gallium nitride (GaN) has emerged as one of the most important technologies enabling the next generation of AI infrastructure. Its ability to switch at higher frequencies while reducing losses makes it particularly well suited to high-density power conversion. According to Scalia, GaN is already becoming dominant within key portions of the AI power stack.

At higher voltages, GaN enables significant improvements in power density compared with traditional silicon MOSFETs. In one example cited by Scalia, replacing silicon devices with GaN enabled a power supply design to achieve approximately 170% greater power density while significantly reducing physical size.

The implications are substantial. As compute modules become denser and cooling requirements intensify, reducing the footprint of power conversion systems becomes increasingly valuable. However, the transition is unlikely to be immediate or universal.

“There is no one solution that fits all,” Scalia says.

Traditional silicon remains highly competitive in lower-voltage applications and continues to benefit from decades of optimisation. Nevertheless, as GaN technologies mature and move into lower-voltage domains, competition between the two approaches is expected to intensify.

Looking ahead, Scalia predicts that GaN will play an increasingly dominant role throughout AI infrastructure, while emerging materials and architectures continue to push performance boundaries even further.

Cooling and power are becoming the same problem
One of the most profound shifts underway is the growing convergence between power delivery and thermal management. In conventional server environments, cooling and power were often treated as separate engineering disciplines. AI is rapidly changing that assumption. High-current delivery, dense packaging and advanced liquid cooling solutions now require power and thermal engineers to work in tandem.

Scalia highlights innovations such as vertical power delivery and three-dimensional power structures that enable shorter electrical paths while simultaneously accommodating advanced cooling solutions. Renesas’ own “tower power” concept, which stacks inductors above power stages within a structure measuring just a few millimetres in height, illustrates how closely packaging and cooling considerations are becoming linked. The objective is not merely to deliver power efficiently, but to do so while minimising inductance, controlling resistance and preserving the transient response required by modern processors.

Even minor parasitic effects can significantly impact performance when systems must respond to current changes measured in thousands of amps per microsecond.

“The packaging is definitely 50% of the entire approach,” Scalia says.

This increasingly integrated view of architecture suggests future innovations may emerge as much from packaging technologies and cooling strategies as from semiconductor devices themselves.

Intelligence moves into the power layer
Another major trend shaping AI infrastructure is the growing role of embedded intelligence and software-defined power management. For decades, power electronics largely relied on fixed-function hardware. Today, digital control enables power systems to become programmable and adaptive. Scalia believes this flexibility will become increasingly valuable as AI infrastructure grows in scale and complexity.

Modern power systems can already be modelled and simulated before processors even exist in hardware. Engineers can create virtual representations of future processors, evaluate power delivery networks and optimise performance well before physical prototypes become available. This digital design methodology significantly reduces development cycles while helping protect extremely expensive AI hardware.

More importantly, programmable power architectures allow infrastructure to dynamically respond to changing workload conditions. Future systems may actively adjust switching frequencies, transient responses and energy flows in real time, using embedded intelligence distributed throughout the power chain. The long-term vision extends beyond the rack itself.

As solid-state transformers and other intelligent power technologies mature, organisations could eventually gain software-defined control extending from medium-voltage grid connections all the way to individual AI processors. Digital power has existed as a concept for decades. AI may finally provide the compelling use case needed to unlock its full potential.

System-level thinking becomes essential
Perhaps the strongest theme emerging from Scalia’s perspective is the importance of system integration.

Historically, semiconductor companies could focus on individual components.

In the AI era, that approach is becoming insufficient. Power, memory, control, packaging, cooling and software increasingly interact in ways that determine overall system performance. Optimising one element in isolation no longer guarantees success. Instead, suppliers are being asked to deliver reference architectures, proof-of-concept systems and integrated design environments that help customers evaluate complex trade-offs across entire platforms.

“The orchestration of the system is what really can make the performance,” says Scalia.

This reflects a broader shift occurring throughout the AI supply chain. Success increasingly depends on close collaboration between semiconductor suppliers, systems vendors, hyperscalers, foundries and software developers. No single participant can solve every challenge independently. As Scalia describes it, innovation is becoming a shared process in which each contributor provides specialised expertise that must be integrated into a unified system.

Looking five years ahead
When considering what AI infrastructure might look like in five years, Scalia offers an answer that focuses less on technology and more on market dynamics. He expects the industry to become significantly more diverse than it is today.

The first phase of the AI revolution was largely driven by a relatively small group of hardware leaders. The next phase, powered by agentic AI and inference, may see software companies, hyperscalers and cloud providers increasingly developing their own custom hardware platforms. Rather than a single dominant architecture, the future may consist of multiple competing approaches tailored to different workloads and deployment models. This diversification will have significant implications for power systems, memory technologies and infrastructure architectures, all of which must support an increasingly heterogeneous AI ecosystem.

“I think the equilibrium of the market will change,” Scalia says. “There will be different platforms and different approaches that will proliferate.”

For infrastructure providers, this means flexibility and adaptability may become every bit as important as raw capability.

The real opportunity beyond compute
The AI industry’s obsession with compute performance is understandable. Accelerators remain the engines driving AI innovation. Yet as deployment scales and inference becomes dominant, attention is shifting toward the supporting technologies that make those accelerators useful. Power delivery, advanced semiconductor materials, memory interfaces, thermal management and intelligent control systems are rapidly becoming strategic battlegrounds in their own right.

In many ways, AI’s future may depend less on how much compute can be added and more on how efficiently entire systems can be orchestrated. As infrastructure moves towards 800V architectures, advanced packaging, software-defined power and increasingly integrated system design, the biggest breakthroughs may come from the technologies operating behind the scenes.

The next decade of AI infrastructure will certainly require more compute. But according to Scalia, the real innovation will occur in the interconnected layers that power, manage and optimise it. And as AI enters its inference-driven era, those layers may ultimately prove just as important as the processors at the centre of the story.