Applications like ChatGPT have become indispensable tools in our lives, requiring vast amounts of data to function normally. As of June 2023, the training dataset for ChatGPT reached 300 billion words, with daily visits of 60 million and over 10 million queries each day, and this is just the beginning. Technologies such as Artificial Intelligence (AI) and High-Performance Computing (HPC) are thriving, and the bandwidth and computing power required by these applications will only continue to grow. Therefore, creating a truly intelligent world requires a scale and complexity of systems that is hard to imagine.
As Moore’s Law slows down, Multi-Die system architecture will provide possibilities for innovative developments across various fields such as GenAI, autonomous driving, and massive data centers. Based on the requirements for power consumption, performance, area (PPA), and more specifically, power consumption, performance, form factor, and cost requirements, current designs are still in a transitional phase from 2D to 3D (and in some applications, even extending to 3.5D).
The future intelligent products will undoubtedly rely on Multi-Die system design, but how can we achieve large-scale adoption in 2024? Here are our four important predictions regarding the design of Multi-Die systems in 2024.
The Adoption of Multi-Die Systems Will Be Phased
Data-intensive applications will be the main driving force behind the packaging of Multi-Die systems such as 2.5D and 3D IC designs. Considering the volume and complexity of data in data centers, we expect that data centers will become the “most enthusiastic users” of Multi-Die systems in the foreseeable future. Other fields, such as mobile communications, are in a transitional phase of utilizing Multi-Die system designs, where the adoption process often opts for more flexibility and requires different stacking technologies and process nodes based on demand.
Industries like automotive typically source components from numerous suppliers, so they may adopt a small chip approach and continue using 2.5D packaging. This method ensures that Electronic Control Unit (ECU) components are readily available and can be easily assembled onto substrates like intermediary layers. In the automotive sector, mixing and matching chips is quite common, and a higher degree of standardization may benefit this practice.
Finding Commonality in Complexity
Vertical integration remains one of the challenges developers must overcome to adopt Multi-Die systems in 2024. HPC data centers have the funding and means to address stacking-related issues, but other industries may not have this capability. If vertical integration is not pursued and the ecosystem is not relied upon, the demand for the smooth operation of Multi-Die systems will sharply increase, posing a significant barrier to their adoption.
The main challenges facing 3D stacking include thermal analysis, power distribution planning, heat dissipation systems, and manufacturing requirements. Despite its complexity, 3D stacking represents the future direction, and the ecosystem must continue to evolve to provide the necessary support. As stacking becomes more prevalent, UCIe, which has become the preferred interface for 2.5D, will see further development in the coming year and beyond. Complexity can be further simplified through two key factors: a common language and clear rules. Components designed in 2.5D or 3D currently use standardized terminology, making it easier to build systems collaboratively with multiple partners.
Having a set of rules and a standard description method is crucial for the success of innovation in Multi-Die systems. Just as vehicles on a highway must obey traffic signs, all forms of chips will continue to be constrained by rules. Standardized testing and reference processes make the stacking process, including manufacturing, clearer and simpler.
Breakthroughs in 3D IC Design: Automation is Key
Current chip architectures are still primarily based on 2.5D, and Multi-Die system design relies on manual processes, with success depending on the alignment between developers’ skills and technical specifications. This may mean that before integration, it is necessary to manually render five, six, or even twenty individual components. In this sense, a truly automated and optimized 3D design with complete architecture has yet to become a reality.
The shift towards automated 3D implementation technology is a crucial step that urgently needs to be taken, helping to accelerate the design process, increase robustness, and ensure that high-performance components do not remain merely conceptual. This transition will continue into 2024 and will encompass all aspects from architectural design and implementation to analysis and verification.
Artificial intelligence can accelerate the optimization of design space. Without AI, the scope of experimentation would be limited; the more bare dies in the process, the more challenging it becomes to plan processes and achieve the best configurations.
Multi-Die Systems Are Not Without Limitations
Despite significant advancements in Multi-Die system design, many limitations still exist. We expect that High Bandwidth Memory (HBM) will continue to serve as external memory in 2024. Depending on bandwidth and heat dissipation requirements, this device may be further positioned closer to the chip in layout, but its off-chip characteristics are unlikely to change.
In terms of stacking, there is technically no upper limit; however, in the current environment, three to four layers are usually the maximum (with a few exceptions). This is not only due to power supply limitations but also manufacturing and reliability considerations. Just like skyscrapers, excessive height may lead to structural issues. High stacking requires more complex manufacturing steps and faces greater risks, and achieving this goal with certainty will take time. It is certain that the industry is working towards this direction. To achieve this goal, a comprehensive embrace of end-to-end automation is necessary.
Conclusion
In 2024, we will further explore the endless potential of Multi-Die systems, leading end-user applications to new milestones. However, the road ahead is not smooth. Overcoming the complexities of Multi-Die system design requires the collective efforts and collaboration of our entire semiconductor ecosystem, working hand in hand to create a future of intelligent connectivity.