Patent analysis reveals high-performance computing semiconductors behind AI boom

AI implementations, such as ChatGPT and autonomous driving systems, have been forefront of everyone’s minds throughout 2023. However, it is crucial to remember that the rise of AI is built on the advances in semiconductor device manufacturing (SDM) to produce high-performance computing devices.

The manufacture of semiconductor devices is a fast-moving industry, with an array of connected technologies. The core element of the field is CMOS logic, which is what drives CPUs and GPUs. In such a rapidly evolving space, it is critical to take a closer look at the manufacturers of this technology and how they are delivering high-performance chips to enable AI innovation.

The top 20 advanced SDM patent owners

TSMC is the clear leader in advanced manufacturing with the largest portfolio size, number of active patent families and the highest Patent Asset Index, which is a measure of the cumulative strength of the patent portfolio. Samsung is second, with ASM International is third.

Across the whole top 20 a series of companies are contributing to the industries in different ways, including:

• TSMC, which is a pure play foundry – a contract manufacturer that solely manufactures for its customers;
• Samsung and Intel, which are hybrids that manufacture their own designs and also those for third parties;
• ASM International and Applied Materials, which are equipment suppliers;
• Western Digital and Infineon, which are integrated device manufacturers (IDM) that only manufacture their own designs; and
• IBM, which is an R&D house that performs R&D but does not manufacture products at scale.

It is worth noting that both IBM and GlobalFoundries appear in this analysis; GlobalFoundries acquired IBM Electronics and its patents in 2014, but IBM still continues its semiconductor R&D.

Figure 2 outlines the type of company, the headquarter location and movement in the field since 2020. While TSMC and Samsung – in the first and second spots respectively – have seen no change, there has been significant movement across the rest of the players with every other organisation shifting rank (eg, Intel rising from ninth to seventh).

Interestingly, a large number of the top players are equipment manufacturers. These companies work extremely closely with manufacturers to develop and implement at scale the machines that facilitate the technology development.

New entrants to the top 20 since 2020 include Beijing Zhiguangxin, which is the parent of Yangtze Memory Technologies and Tsinghua Unigroup. Along with Ruili IC, these make up two of the three companies based in China that are in the 2023 ranking, neither of which were in the 2020 ranking. It is unclear how the US CHIPS Act might affect the future development of these and other Chinese players. Another new addition to the top 20 is DISCO, a Japanese equipment supplier.

CMOS logic for tomorrow’s high-performance computing

Achieving lower-power and higher-performance computing demands more than just improvement; revolutionary innovation in the core semiconductor manufacturing needs to be realised. The transistors at the heart of CMOS logic need to develop to allow for shrinking down to as low as 2nm – the width of a strand of DNA. At that scale, simply shrinking the device dimensions is no longer sufficient. Semiconductor device manufacturers are running into the physical limitations of atom size and effects at the quantum scale. To overcome such challenges, tremendous amounts of effort and resources have been invested in innovation when it comes to the designs and structures of the transistors over the past decade.

Up until 2012, transistors were planar devices – a simple, flat transistor channel structure under a gate – as illustrated on the far left in Figure 3. Then, in 2012, Intel launched the first commercially available 22nm 3D transistors called “FinFET” after the fin-shaped channel structure. For FinFET devices, the gate covers three sides of the fin channel surface to have a higher current and to overcome issues of planar structures with smaller channel widths. Since then, TSMC has emerged as the market leader by producing the most advanced FinFET devices for major CMOS logic product companies, such as Nvidia, Apple and AMD.

After more than 10 years, scaling FinFETs has reached its performance limits at the 3nm process node and an even more advanced transistor type is needed: the gate-all-around (GAA) transistor, which is pictured on the right-hand side in Figure 3. GAA transistors have channels that are fully surrounded with the gate. GAA transistors provide an even better performance then FinFET and also permit further reduction in device dimensions.

Figure 4 highlights the portfolio sizes of the four major semiconductor logic device manufacturers (plus IBM) for these two transistor types, updated from our previous 2021 article (see “Patent data shows the complexity of the semiconductor manufacturing market”). It is clear that GAA is the more recent innovation of the two with the most significant increases in the past five years, whereas for FinFET, there has been clear development for over a decade.

TSMC holds the number one position in both of these technologies today. Samsung, IBM and Intel appear to have also continued developing in the area, but at a much lower rate. GlobalFoundries's initial high portfolio size is a result of its purchase of IBM, but the slow-down seems to correlate with its announcement that it was dropping the development of a 7nm process technology in 2018.

For GAA, there was no clear leader before 2018. Between 2018 and 2020 IBM appeared to have increased its portfolio, but this development trend did not continue at the same pace in the following years. There has been a noticeable increase from Samsung and Intel, but this is dwarfed by TSMC’s rapid increase, which grew in a factor of four from 2020 to 2022.

Figure 5 shows the current position of these owners in GAA in terms of quality measured by competitive impact on the vertical and quantity measured by portfolio size on the horizontal. The darkest bubble is the status in 2023 with each lighter bubble stepping back in time from 2023 to 2022, 2020, 2018, 2016 and 2014.

TSMC leads both in portfolio size and strength today. Since 2020 TSMC's portfolio strength has shown a significant increase in bubble size. Further, Samsung, IBM and Intel have all increased both the size and quality of their portfolios. The average patent quality of these four companies improves as their portfolio size increases, which means that the newly filed patents must be of the same or higher quality than the previous patents. This is usually not the case for growing portfolios and reflects the advancement of GAA technology.

The only outlier is GlobalFoundries, whose portfolio size has hardly changed since 2018 despite the increase in average quality. This reflects the company’s choice to not pursue GAA technology.

Industry news indicates that Samsung has recently started its GAA production at the 3nm process node, reportedly reserving initial production for cryptocurrency miners. Both TSMC and Intel announced that they would start their GAA production at the 2nm process node in next couple of years.

Augmenting transistor innovation — advanced packaging

Moore’s Law is the observation that the number of transistors on an integrated circuit will double every two years with minimal rise in cost. Historically, CMOS logic chips have all of their functions integrated onto the single system on chip (SoC), such as computing cores, memory and input/output controllers. With the demand of more functionality of modern semiconductor devices, additional devices (eg, high-bandwidth memory), are integrated with logic chips by advanced packaging technology.

In high-performance computing, for example, processors require large amounts of close-by memory with short interconnections and high data transfer rates. This near-memory computing requirement is fulfilled by advanced packaging, like Intel's embedded interconnect bridge and Foveros. Nvidia's H100 is a commercialised example of advanced packaging, which builds on TSMC’s CoWoS technology. Another example is AMD's MI300, which uses TSMC’s CoWoS and system on integrated chips (SoIC). AMD’s CEO Lisa Su even stated that AMD "couldn't have done [MI300] without TSMC".

Figure 6 shows the top 10 players in advanced packaging. TSMC is clearly dominating this field, followed by Samsung and Intel. The other players are much smaller, but also have a slightly different technology focus. Adeia, Micron, Beijing Zhiguangxin Holding, Western Digital and Infineon are not focused on CMOS logic technology. MediaTek and Qualcomm are fabless patent owners.

The right side of Figure 6 highlights the development of the top three players. As advanced packaging for CMOS logic depends on advanced semiconductor device manufacturing capabilities, these three companies are capable of high-volume advanced packaging to meet the AI boom. Both Intel and TSMC have had a positive development in recent years. Impressively, TSMC’s larger size and higher-quality grants it the number one spot. There is a slight downward trend in quality from Samsung as it has grown its portfolio size.

Extending Moore’s Law

As the demands of these individual functions, the IP blocks increased, as did the size of the circuitry required for these functions. The semiconductor chips for these devices became so large so as to not be easily manufacturable. The innovative solution is to manufacture such IP blocks on individual smaller chips, or ‘chiplets’, and to recombine them using advanced packaging. This approach also allows adding and connecting computing units’ chips and/or chiplets with other external memories and neural network computing units to offer even more functionality and performance.

For high-performance computing, even shorter interconnection and higher data transfer rates between chips and chiplets are necessary. Connecting chiplets with interconnects spaced apart only 20 micrometer or less is cutting-edge advanced packaging and is called fine-pitch packaging. TSMC's SoIC or Intel's Foveros Direct can achieve this to enable high-performance computing. Players (eg, AMD) are already bringing this to market in 3D V-Cache technology, which is made by TSMC.

Figure 7 reflects the owners in this space and the development of their portfolios overtime. TSMC set an early lead in this cutting-edge technology and has continue to develop over the last decade. Other players, however, have not seen the same level of continued development. In recent years there has been a clear increase from Intel as it begins its development; Intel has a very positive trend today but is still well behind TSMC’s position.

Looking to the future

The march for innovation is relentless – nowhere more so than in advanced semiconductor CMOS logic device manufacturing and advanced packaging. These two technologies have evolved in parallel in the past, but now the boundaries between them are dissolving. The co-development of advanced CMOS logic and advanced packaging will be key to enable the next generation of high-performance computing devices.