Intel’s 18A node gets most of the spotlight recently – with an ongoing battle between TSMC’s and Intel’s management teams on the merits of TSMC N2 vs Intel’s 18A. However, it is 14A that will be the make-or-break node for Intel Foundry. Winning customers starts with process technology, and Intel is betting big here, but they need a generation where everyone gets comfortable. Customers will use 18A to dip their toes in the Intel waters with less critical chips that are not core to their business; if all goes well, they will look to 14A as the main process for their linchpin designs – think the ..read more

Nvidia’s announcement of the B100, B200, and GB200 has garnered more attention than even iPhone launches, at least among the nerds of the world. The real question that everyone is asking is, what is the real performance increase? Nvidia’s claimed 30x, but is that true? Moreover, the question is really, what is the performance/TCO? In the last generation, with the H100, the performance/TCO uplift over the A100 was poor due to the huge increase in pricing, with the A100 actually having better TCO than the H100 in inference because of the H100’s anemic memory bandwidth gains and massive price inc ..read more

Before Pat Gelsinger took over Intel as CEO, the company spent over a decade in a slow descent due to a focus on financial engineering. The decline was set in motion by the then CEO, Paul Otellini, who made the shortsighted decision to turn down the iPhone contract due to apprehension over margins. The main concern was that Apple's customization demands would be costly and would be amortized over low volume projections that turned out to be woefully underestimated by Intel. This led to Intel missing out on the last decade’s largest area of growth: mobile. Intel’s own assessment of its proces ..read more

At GTC, Nvidia announced 8+ different SKUs and configurations of the Blackwell architecture. While there are some chip level differences such as memory and CUDA core counts, most of the configurations are system level such as form factor, networking, CPU, and power consumption. Nvidia is offering multiple 8 GPU baseboard style configurations, but the main focus for Nvidia at GTC was their vertically integrated DGX GB200 NVL72. Rather than the typical 8 GPU server we are accustomed to, it is a single integrated rack with 72 GPUs, 36 CPUs, 18 NVSwitches, 72 InfiniBand NICs for the backend networ ..read more

This post was published on 3/18 and then taken down due to moral reasons. It is republished on 3/23 with some content removed. We stand by the analysis regarding margin. Nvidia announced their new generation of Blackwell GPUs at GTC. We eagerly await the full architecture white paper to be released to detail the the much needed improvements to the tensor memory accelerator and exact implementation of new MX number formats, discussed here. We discussed many of the high level features of the architecture such as process node, package design, HBM capacity, SerDes speeds, here, but let’s dive a bi ..read more

The gold rush for AI infrastructure is creating huge opportunities for the companies supplying enabling technologies. Not everyone is an Nvidia in this infrastructure build out bonanza, there are many small key players too. Today we’ll dive into Astera Labs, whose chips have been silently shipped in more than 80% of AI servers. Astera Labs is a datacenter connectivity pure-play and targets mainly 3 customer types: hyperscalers, AI accelerator vendors, and system OEMs. Astera Lab’s product portfolio is currently comprised of 3 families: Aries retimers, Taurus active electrical cable (AEC) paddl ..read more

If we travel back 2 years in time, before the rapid rise in AI, much of the datacenter hardware world was chasing CXL. It was promised as the messiah to bring heterogenous compute, memory pooling, and composable server architectures. Existing players and a whole new host of startups were rushing to integrate CXL into their products, or create new CXL based products such as memory expanders, poolers, and switches. Fast forward to 2023 and early 2024, and many projects have been quietly shelved and many of the hyperscalers and large semiconductor companies have almost entirely pivoted away. With ..read more

The boom in demand for AI clusters has led to a surge in focus on datacenter capacity, with extreme stress on electricity grids, generation capacity, and the environment. The AI buildouts are heavily limited by the lack of datacenter capacity, especially with regard to training as GPUs need to be generally co-located for high-speed chip to chip networking. The deployment of inference is heavily limited by aggregate capacity in various regions as well as better models coming to market. There is plenty of discussion on where the bottlenecks will be – How large are the additional power needs? Whe ..read more

Hybrid bonding is going to be the most transformative innovation to semiconductor manufacturing since EUV. In fact, it will have an even bigger impact on the design process than EUV itself, branching from package architecture down to cell design and layouts. The IP ecosystem will be dramatically reshaped, but so will manufacturing flows. The era of shrinking transistors in 2D will continue, but at a muted pace, but hybrid bonding will bring in a new age, where chip designers think 3D. With that hype filled ballad finished, we should note there are many major engineering and technical challenge ..read more

Today’s post is heavily focusing on number formats, but through the rest of the month we have 3 deep dives that focus on both technology, the market, and future architecture/purchasing on hybrid bonding for chip design and economics of chiplets, physical datacenter infrastructure and the power problem, and Amazon’s AI silicon strategy and volumes from Alchip and Marvell. Quantization has played an enormous role in speeding up neural networks – from 32 bits to 16 bits to 8 bits and soon further. It’s so important that Google is currently being sued for $1.6 billion to $5.2 billion for allegedly ..read more