TL;DR
When I saw the news, I was a bit shocked to find out the 20B was in dollars, not model parameters. Looking at it from the perspective of Scaling Law, from Mellanox's 6B deal a few years ago to this 20B deal. Maybe I should train a 671B (same as Deepseek-V3's parameters) business and have Jensen Huang buy it? Just joking, let's take a look at why Groq is worth so much.
The structure of this article is as follows:
1. Background
1.1 Has the story of integrated training and inference changed?
1.2 Analysis from a business model perspective
1.3 From a technical perspective
2. Groq Arch Overview
2.1 What is Software Defined?
2.2 Deterministic Design
2.2.1 Chip-level Determinism
2.2.2 System-level Determinism
3. Groq TSP Architecture
3.1 TSP Microarchitecture
3.2 Software Abstraction
3.3 ISA
3.4 MXM
3.5 VXM
3.6 SXM
3.7 MEM
4. System Architecture
4.1 Scale-Out Topology Choice
4.2 System Hierarchical Packaging
4.3 Inter-chip Links and Flow Control
5. Maintaining Determinism in a Distributed System
5.1 Overview
5.2 HAC
5.3 Initial Program Alignment
5.4 Runtime Deskew for Resynchronization
6. Software-Scheduled Network (SSN)
6.1 A Priori Knowledge of Traffic Patterns
6.2 Scheduling, not Routing
6.3 Deterministic Load Balancing
6.4 Flow Control
7. RAS
7.1 FEC and Software Replay Mechanism
7.2 ECC
8. Groq Software Architecture
8.1 GroqIt
8.2 Groq Compiler
8.3 Groq Runtime
9. Future Evolution
9.1 Comparison with GPGPU Architecture
9.2 Groq Roadmap
10. How NVIDIA can absorb Groq's technology
10.1 NVIDIA GPGPU Problems
10.2 Is Determinism Really Necessary?
10.3 CUDA Core Evolution
10.4 Compiler Perspective
10.5 3D Architecture
10.6 On Interconnection
10.7 From a Reliability Perspective
TL;DR
When I saw the news, I was a bit shocked to find out the 20B was in dollars, not model parameters. Looking at it from the perspective of Scaling Law, from Mellanox's 6B deal a few years ago to this 20B deal. Maybe I should train a 671B (same as Deepseek-V3's parameters) business and have Jensen Huang buy it? Just joking, let's take a look at why Groq is worth so much.
The structure of this article is as follows: