Qualcomm Takes On The AI Inference Memory Bottleneck With Working HBC Silicon

Memory bandwidth is described as a central constraint for LLM inference because every generated token requires model data to be fed to the compute units, and at data center scale the energy and latency of that movement affect throughput, power consumption and total cost of ownership. HBC aims to address this by placing compute closer to memory to cut data movement and raise effective bandwidth.

The HBC design stacks low-power LPDDR dies on top of an active AI accelerator base die using wafer-to-wafer bonding, shrinking the distance data travels from millimeters to micrometers. As reported by HotHardware, this approach results in an 18x increase in effective memory bandwidth over LPDDR5X and more than 5x the bandwidth per watt of projected HBM4 specifications, though the efficiency improvements landed slightly below the 6x benchmark Qualcomm had projected earlier. Qualcomm's own footnote says the figures are estimates based on measured silicon results normalized to target operating conditions and compared with published competing product specifications, and that final performance may vary by implementation, workload and system configuration.

Qualcomm describes a staged program spanning architecture, process development, test vehicles, silicon validation and product integration, and says it has worked with foundry and assembly partners to qualify wafer-to-wafer bonding. To address defects in its stacked layers, the design includes redundancy and remapping, allowing it to work around faulty memory. The company points to more than a decade of memory-on-logic work, including commercialized direct bonding of DRAM onto logic and a hand in shaping JEDEC LPDDR standards, and commercial products from Qualcomm are expected to employ a four-layer DRAM stack, up from a two-layer one.

The Dragonfly AI250 is targeted for sampling in the first half of 2027 and will slot into Qualcomm's Dragonfly Rack inference infrastructure. The hardware includes a full liquid-cooled rack with 38.6TB of memory and 7.4 petabytes per second of effective bandwidth, and Qualcomm plans to deploy its Oryon-based C1000 server CPU in 2027, followed by next-generation HBC in the Dragonfly AI300 in 2028.

It is worth keeping in mind that these results are from Qualcomm's own lab based on early silicon, and there is a possibility that independent benchmarks paint a different picture, which the outlet says it will update readers about in the future.