If the quickening pace of change in modern life was ever in doubt, the breakout of generative artificial intelligence – already reduced to ‘GenAI’ – and the success of ChatGPT have surely put it to rest. Scalability or virality is one of the most desired attributes of a business, especially if accompanied by minimal asset intensity. Best described as ‘network effects’, the advent of the internet and social media have allowed this concept to proliferate and morph in ways that sometimes boggle the mind.

Take The Stanley Thermos for instance. I was amused to read about its viral resurgence from a dusty attic relic to an Instagram darling¹). Similarly, in the UK, an ordinary home cleaning product became a coveted possession almost overnight. One woman ‘cleanfluencer’ unknowingly propelled an obscure brand, The Pink Stuff, onto shelves at Walmart and Amazon². 

These are not software products or video games, which have an innate quality of achieving scale if conditions are right. Making physical stuff is hard to do at scale; ensuring quality and consistency, managing supply chains, and more. 

These trendy comebacks reveal a lesson for investors in Asia. We do not have the ‘Mag Seven’; yet the real unsung heroes of this shift driven by AI are some of Taiwan's hardware manufacturing businesses. TSMC is the one that readily comes to mind, but there are several other smaller companies which are critical to the success of this rapidly growing technology. 

To understand what this changing landscape means for the manufacturing of chips and servers, I prompted Gemini, Google’s AI app, with this question.

As GPU's become the workhorses of AI, from a manufacturing standpoint, what are the key challenges to producing AI chips and AI servers? Focus on the complexity or difficulty in manufacturing as well as areas that are driving changes to form factors or design approaches for AI servers as compared to normal servers.

This was the response:

For AI Chip Manufacturing:
(A)    Complexity and Cost:
Advanced architectures: AI chips often employ specialised architectures like tensor cores, which are more intricate to design and fabricate compared to standard CPU cores. This complexity translates to higher manufacturing costs and lower yields (usable chips per wafer).

Smaller feature sizes: Pushing performance boundaries means using ever-smaller transistors, requiring expensive and cutting-edge lithography techniques (eg EUV) that aren't readily available at scale.

Memory integration: Integrating high-bandwidth memory (HBM) directly onto the chip improves communication but significantly increases fabrication complexity and cost.

(B)    Heat dissipation:

High power consumption: AI computations demand significant power, leading to intense heat generation. Traditional packaging and cooling solutions might not suffice, requiring innovative thermal management methods.

For AI Server Manufacturing:
(A)    Scalability and efficiency:

Heterogeneous architectures: Combining different chips (CPUs, GPUs, specialised AI accelerators) within a server demands new designs for efficient communication and data flow.

Power and cooling infrastructure: Accommodating high-power AI components necessitates robust power delivery and intricate cooling systems, impacting server design and layout.

Software optimisation: Server hardware needs to be tightly coupled with optimszed AI software to ensure efficient resource utilisation and minimise bottlenecks.

(B)    Changes and New Designs:

Modular servers: Moving away from rigid, fixed setups to modular designs allows for flexible configurations based on specific AI workloads, improving efficiency and scalability.

Liquid cooling: Traditional air cooling might not be enough for high-density AI servers. Liquid cooling systems, while effective, pose challenges in terms of cost, complexity, and potential leaks.

Specialised interconnects: New high-bandwidth, low-latency interconnect technologies are needed to facilitate efficient communication between diverse processing elements within the server.

What this is telling us – I apologise for quoting at such length but I think it’s interesting – is that there is a great deal of development still to come if we are to reap the benefits of AI in ways that are meaningful and significant. Asian economies are likely to benefit from this process in ways and to an extent that is not yet recognised, in my view, by international investors. 

We own several companies across this spectrum of innovation and change in manufacturing. One of them is Jentech Precision Industrial. With poor analyst coverage, it was clubbed under machining and casting businesses. They were seen as no more than another component manufacturer in a long list of components in a very cyclical industry.

Yet, changes around AI servers lead to significant added value and potentially an expanded business cycle. The simplest way to understand this is the likelihood of a PC/device replacement cycle for companies and consumers driving volumes. Additionally, the monstrous increase in average selling price (ASPs) for AI related critical components. Though difficult to cite averages, suffice to say there could be an almost 8-10x increase in realisations for some component makers compared to normal server components.

Jentech crafts intricate ‘heat spreaders’ that keep AI servers cool under pressure. Their products help dissipate heat from integrated circuits (IC) as well as strengthen the thermal module while avoiding warpage of the IC when it heats up. The feature that defines companies like Jentech is years of manufacturing knowhow. Critical equipment is self-designed but manufactured in Japan; another secret to competitiveness is longer durability of their moulds which reduces costs over the long run. As their client AMD moved to larger-sized heat spreaders, Intel, lagging AMD, had no choice but to approach Jentech too. Now, Jentech's thermal solutions play an important role in data centres powering AI breakthroughs and find uses in self-driving cars. Think of it this way - while Nvidia stole the limelight, Taiwanese companies like Jentech were busy building the props and the stage machinery, ensuring the show went on without a hitch. 

These manufacturing companies are the shovel-makers for the Nvidia goldmine. My knowledge on Nvidia is shallow, but from what I read, most of their clients want to ensure they are not solely dependent on Nvidia, which only means the rest of the Magnificent Seven could become potential customers for these Taiwanese companies in time. Nothing in technology can be taken for granted but many of these Taiwanese firms are the go-to partners of choice with very few alternatives when it comes to leading edge technologies.

Source:

[1] https://www.nytimes.com/2022/05/17/style/stanley-tumbler.html (subscription needed)

[2] https://www.nytimes.com/2024/02/11/business/tiktok-cleaning-pink-stuff.html