Digital Bear Talks Tech: Groq Secures $750 Million in Funding! What Makes This 14nm Chip Dare to Challenge NVIDIA?Friends, today Digital Bear wants to share some hardcore tech news with you! The AI chip industry has been buzzing these past few days—Groq has just announced the completion of a new funding round of $750 million, with a valuation skyrocketing to $6.9 billion! You might be wondering, “What’s the background of this company? How can they be so formidable?” Don’t worry, Digital Bear will dig into their “backing” and black technology.First, let’s talk about the “investors” behind this funding round. The lead investor is from Dallas,Disruptive, a company known as the “golden touch” in the investment world, having invested in renowned companies like Airbnb, Spotify, and Slack, contributing nearly $350 million to Groq alone! Even more astonishing, giants like BlackRock, Deutsche Telekom Capital, Samsung, and Cisco have also joined in, making the lineup so impressive that Digital Bear can’t help but think, “Groq must be up to something big!”
Founder Jonathan Ross boldly stated, “Inference is defining the era of artificial intelligence. What does this mean? Simply put, AI must not only learn (train) but also quickly solve problems (infer). For example, if you ask ChatGPT a complex question and it can respond instantly, that experience would be top-notch, right? Groq’s goal is to make AI inference both fast and cost-effective, and their secret weapon is theLPU chip.Speaking of LPU, it is Groq’s “trump card.” Although it uses a14nm process (while NVIDIA’s H100 is at 4nm), they are innovating in architecture! Digital Bear highlights:Groq’s self-developed TSA architecture allows the chip to handle millions of data streams simultaneously, with an80TB/s on-chip memory bandwidth that directly crushes traditional GPUs. Official data shows that the LPU’s computing power reaches1000 TOPS, making it 10-100 times faster than GPUs and TPUs on certain models!For a practical example: using Groq’s cloud server to run the Llama2 or Mistral models,it can generate 500 tokens per second, while ChatGPT-3.5 only manages about 40. Even more impressive is the power consumption—generating the same results, NVIDIA GPUs require 10-30 joules, while Groq only needs 1-3 joules,cutting energy costs down to one-tenth! At this rate, Groq’s claim of “surpassing NVIDIA in three years” doesn’t seem like an exaggeration.Of course, going solo is not as powerful as a “group attack.” Groq has connected576 LPU chips into a cluster using fiber optics, boosting the generation speed of the Llama2 70B model toover 300 tokens per second, which is 10 times faster than 8 H100 GPUs, with even lower power consumption! This operation is simply elevating cost-performance to new heights, which is why even Saudi Arabia has invested $1.5 billion to support them in building data centers.At this point, some friends might ask, “Digital Bear, can this technology be implemented?” Don’t worry! Groq has already provided services to2 million developers and several Fortune 500 companies, with data centers in North America, Europe, and the Middle East expanding. They have even opened a demo online, supporting popular models like Mistral’s Mixtral8x7B and Llama2, so interested parties can experience it firsthand!Finally, let’s talk about something practical. The Groq team consists ofthe original team behind Google TPU, with the founder being a core designer of TPU and the hardware VP being a big shot from AWS and Intel. With such an “all-star lineup,” combined with the frenzied bets from the investment and industry sectors, Digital Bear believes the landscape of the AI chip market is truly about to change.So here’s the question: If Groq can indeed achieve crushing inference speeds, do you think it will become the next NVIDIA? Let’s chat in the comments! (By the way, I heard they even secured funding from Saudi billionaires, making this plot twist more exciting than the TV show “Silicon Valley”!)