Groq

Groq builds infrastructure for AI inference, the work of running trained models. Its products include the LPU Inference Engine and GroqCloud, which provides model access through an API. In December 2025, Groq licensed its inference technology to Nvidia under a non-exclusive agreement while continuing as an independent company. By June 2026, it had concentrated its strategy on operating an inference cloud and reported 13 data centers across four regions.[2][6][7]

That cloud business followed earlier work on inference processors. Founded in 2016, Groq identifies Jonathan Ross, an inventor of Google's Tensor Processing Unit, as its founder.[5][2]

The June 2022 paper Answer Fast: Accelerating BERT on the Tensor Streaming Processor explains an inference optimization: fusing nonlinear operations with matrix multiplication to keep the processor's on-chip matrix units efficiently occupied. Authors Ibrahim Ahmed, Sahil Parmar, Matthew Boyd, Michael Beidler, Kris Kang, Bill Liu, Kyle Roach, John Kim, and Dennis Abts reported deterministic tail latency of 130 microseconds for batch-1 BERT-base inference, which they described as six times faster than the then-current state of the art. The result concerned that BERT experiment, rather than a general language-model generation rate.[8]

Groq's token-generation speed drew attention in early 2024. In Latent Space's March recap, Alessio described a viral demonstration running Mistral at approximately 500 tokens per second. The discussion also raised a commercial question: whether enough concurrent demand could sustain the utilization needed to justify the hardware investment. swyx credited spreadsheet work to himself and Dylan Patel and cautioned that his favorable cost estimate depended on demanding parallelism and utilization assumptions.[1]

GroqCloud soft-launched on February 19, 2024. On March 1, Groq announced its acquisition of Definitive Intelligence, whose team had already worked with Groq to build the developer playground, including integrated documentation, code samples, and self-serve access. Definitive Intelligence co-founder and CEO Sunny Madra was appointed to lead the GroqCloud business unit, initially focusing on capacity, efficiency, partnerships, and the developer platform. Groq reported thousands of developers actively using its API by the announcement.[2]

The acquisition also supplied engineering resources to formalize Groq Systems, a separate unit serving public-sector customers and organizations requiring hardware for AI compute centers. Customers could purchase LPU-powered systems or access the LPU Inference Engine through GroqCloud's API by buying Tokens-as-a-Service. The two units gave Groq distinct hardware and cloud delivery channels.[2]

In August 2024, Axios reported that Groq raised $640 million in Series D funding at a $2.8 billion valuation, led by BlackRock funds.[3]

Groq reported bringing a Saudi Arabian inference cluster online in eight days in December 2024 and delivering GroqCloud services to customers worldwide from Dammam. On February 10, 2025, it announced a $1.5 billion commitment from Saudi Arabia for expanded delivery of LPU-based inference infrastructure. This was an infrastructure commitment, rather than an equity financing round. At LEAP 2025, Jonathan Ross, Tareq Almin, and Saudi Aramco CTO Ahmad O. Al-Khowaiter demonstrated reasoning models, the Saudi-created Allam model, and English and Arabic text-to-speech running live.[4]

On September 17, 2025, Groq announced $750 million in new financing at a $6.9 billion post-money valuation. Disruptive led the round, with significant investment from BlackRock, Neuberger Berman, Deutsche Telekom Capital Partners, and an unnamed US-based West Coast mutual fund manager. Groq then reported more than two million developers and existing data centers in North America, Europe, and the Middle East.[5]

On December 24, 2025, Groq announced a non-exclusive licensing agreement with Nvidia. Ross, Groq president Sunny Madra, and other team members would join Nvidia to advance the licensed technology. Groq said it would remain independent, with Simon Edwards stepping into the CEO role and GroqCloud continuing without interruption.[6]

Nvidia subsequently introduced Groq 3 LPX, a rack-scale accelerator for its Vera Rubin platform. In their March 2026 technical explanation, Kyle Aubrey and Farshad Ghodsian describe 256 interconnected LPUs, each with 500 MB of primary on-chip SRAM. The compiler explicitly schedules computation, data movement, and synchronization. Larger models span multiple LPUs because each chip's SRAM capacity is finite; these specifications apply to Groq 3 LPX.[9]

LPX divides inference work with Rubin GPUs. GPUs process the prompt during prefill and perform attention during token generation, while LPUs accelerate latency-sensitive feed-forward operations and mixture-of-experts execution. Nvidia Dynamo coordinates the repeated exchange of intermediate activations between the GPU and LPU backends.[9]

On June 22, 2026, Groq announced $650 million in growth capital, led by Disruptive and Infinitum. Following the Nvidia license, Groq's board, lead investors, and management had concentrated its strategy on building an inference cloud. The capital would fund fit-out of its existing footprint with newer inference technology, including Nvidia LPX. The announcement identified Alex Davis as chairman, Adam Winter as CEO, Matt Eng as CFO, and Alan Rice as COO.[7]

As of that June announcement, Groq reported 13 operating data centers across North America, Europe, the Middle East, and Asia-Pacific, serving more than five million developers and thousands of AI-native companies consuming trillions of tokens weekly. The developer total did not specify active users or paying customers. Groq expected to scale toward 200 MW by the end of 2027, a future capacity target.[7]

Nvidia announced Groq 3 LPX was in full production on August 24, 2026. It identified Nebius as the first adopting AI cloud and said Groq planned to be among the platform's earliest adopters after Nebius. Groq's LPX adoption remained a plan in that announcement.[10]