Together AI

Together AI builds cloud services for training, fine-tuning and running generative AI models, alongside research in datasets, model architectures and GPU efficiency. Its products span hosted APIs, GPU clusters and custom-model development with customer ownership. By March 2026, its research and product program also covered reinforcement learning and infrastructure for agentic workflows.[5][7][13]

These services grew from a company started in June 2022. Its early work on distributed computing and open training data supplies the context for the cloud platform it introduced in 2023.[1][5]

Together’s co-founders were Vipul Ved Prakash, Ce Zhang, Chris Ré and Percy Liang. In a February 2024 interview, Prakash dated its founding to June 2022 and described its initial purpose as a platform for “open source, independent, user-owned AI systems.” He recalled that Ré introduced him to Zhang and that, within their first five minutes of conversation, he decided they should start the company. Prakash and Zhang identified themselves in that interview as CEO and CTO, respectively.[1][3]

The initial technical exploration concerned GPUs scattered across the internet. Prakash explained that reducing communication during gradient averaging could make cheaper compute across networks usable for model building. Zhang connected this work to a broader systems problem: moving data between machines in distributed computing and between levels of GPU memory in attention computation.[1]

Together announced a $20 million seed round led by Lux Capital on May 15, 2023. By then, it had released GPT-JT, OpenChatKit and RedPajama and built a specialized cloud platform using distributed-optimization research; public access to that platform was still planned for the coming months.[3]

On April 17, 2023, Together and collaborators released RedPajama’s pre-training dataset, comprising 1.2 trillion tokens prepared by following LLaMA’s published recipe. This reproduced the recipe rather than releasing Meta’s original corpus. The project brought together Together, Ontocord.ai, ETH DS3Lab, Stanford CRFM and Hazy Research, with Université de Montréal and LAION among the acknowledged dataset participants. It also received INCITE computer time and used Oak Ridge Leadership Computing Facility resources.[4]

The dataset combined CommonCrawl, C4, GitHub, arXiv, books, Wikipedia and StackExchange, with preprocessing and quality filters made openly available. Pre-training data was the first stage of a project that also encompassed base models and instruction-tuning data and models. Zhang credited earlier community datasets, including The Pile and C4, as precedents; Cerebras subsequently built SlimPajama by deduplicating RedPajama data.[4][1]

RedPajama-Data-v2, released October 30, 2023, changed the emphasis from a particular training recipe to configurable filtering and weighting. Together reported a 30-trillion-token filtered and deduplicated subset drawn from more than 100 trillion raw tokens across 84 CommonCrawl dumps in English, French, Spanish, German and Italian. More than 40 precomputed quality annotations let developers assemble their own pipelines using signals for repetition, text characteristics, content, domain similarity and deduplication.[6]

Together’s July 14, 2023 product announcement introduced two complementary services. Together API offered fine-tuning and private inference endpoints; Together Compute paired high-end GPU clusters with a distributed training stack for groups pre-training models on their own data. Together reported hosting more than 50 models at the time, including RedPajama, LLaMA, Falcon and Stable Diffusion. Together Compute was renamed Together GPU Clusters on November 5, 2023.[5]

Together Custom Models, announced November 13, 2023, added expert assistance across data discovery and optimization, architecture and training recipes, training, tuning and alignment, and evaluation. Its ownership policy gave customers the resulting model and all checkpoints, with freedom to run it elsewhere. The launch included Arcee’s in-progress patent-model project: a 7-billion-parameter model training over 4 billion tokens.[7]

Data selection was part of this service. DSIR estimates importance weights in a reduced feature space and then resamples data according to those weights. Zhang explained the practical idea as selecting corpus material resembling a validation set or a useful reference domain, such as Wikipedia. In the February 2024 interview, he attributed the DSIR paper to Liang rather than presenting it as a Together invention.[7][1]

By February 2024, Together offered serverless inference and fine-tuning, while serverless training remained a development direction. Its disaggregated cloud placed a common service layer over infrastructure differing in geography, reliability and hardware. In March 2024, Together reported working with more than 10 GPU cloud platforms; named network participants included Crusoe Cloud, Applied Digital, Lambda Labs, Vultr, Oracle Cloud and ClusterPower.[1][9]

Tri Dao joined Together as Chief Scientist in July 2023. His FlashAttention work had first been published in May 2022, before Together’s founding. FlashAttention computes exact attention with linear memory requirements while retaining quadratic arithmetic. Kernel fusion, tiling and online softmax reduce traffic between large, slower GPU memory and small, fast on-chip memory. Dao credited his co-authors and earlier systems and softmax techniques for this work.[2][11]

Dao’s FlashAttention-2 paper, submitted July 17, 2023, reduced non-matrix-multiplication operations, spread attention computation across GPU thread blocks and repartitioned work between warps to reduce shared-memory communication. It reported approximately twice the attention speed of FlashAttention on A100 GPUs. Dao described NVIDIA’s CUTLASS 3 library as the starting point for the refactoring that became FlashAttention-2 and credited the CUTLASS team’s support.[11][2]

Together also explored alternatives to pure Transformer architectures. StripedHyena combines rotary grouped attention with gated convolutions in Hyena blocks: Hyena layers perform most sequence processing, while attention supplements targeted pattern recall. Its December 2023 software citation credits Michael Poli, Jue Wang, Stefano Massaroli, Jeffrey Quesnelle, Ryan Carlow, Eric Nguyen and Armin Thomas. Nous Research collaborated on the StripedHyena-Nous-7B chat model. Together later collaborated with Arc Institute on Evo, a biological generative model based on StripedHyena for DNA, RNA and proteins.[12][9]

In February 2024, Prakash described publishing broadly useful work such as FlashAttention and Flash-Decoding while retaining some inference-stack components for competitive price-performance. Closed components were discussed internally and could be reconsidered later. He also set out a research ambition of 5,000 tokens per second, arguing that faster generation could serve machine-consumed outputs, increase GPU throughput and enable immediate-response interfaces; this was a target, not a reported achieved rate.[1]

On November 29, 2023, Together announced $102.5 million raised in Series A financing, led by Kleiner Perkins with NVIDIA and Emergence Capital participating. Bucky Moore was to join its board. The company described infrastructure growing toward 20 exaflops across US and EU data centers. In the February 2024 interview, Prakash reported approximately 7,000–8,000 GPUs, mostly A100s and H100s, and a 38-person team with roughly 40–45% dedicated to research.[8][1]

On March 13, 2024, Together announced $106 million raised in a new financing round, led by Salesforce Ventures with Coatue participating. It reported more than 45,000 registered developers for its serverless inference and fine-tuning APIs. That figure counted developer registrations, not paying customers.[9]

Together’s February 20, 2025 announcement described a $305 million Series B led by General Catalyst and co-led by Prosperity7, valuing the company at $3.3 billion. It reported more than 450,000 AI developers and support for more than 200 models. Expansion included 200 MW of secured power capacity and a Hypertec partnership to co-build a 36,000-GPU NVIDIA GB200 NVL72 cluster—commitments and construction plans rather than an operational fleet count. Together also announced that it had recently acquired CodeSandbox, with plans to integrate code interpretation.[10]

At its first AI Native Conf on March 5, 2026, Together announced seven research and product releases. Among them, FlashAttention-4 addressed Blackwell’s uneven hardware scaling: tensor-core throughput increased faster than shared-memory bandwidth and exponential units. Ted Zadouri, Markus Hoehnerbach, Jay Shah, Timmy Liu, Vijay Thakkar and Tri Dao combined asynchronous matrix-multiplication pipelines, larger tiles, software-emulated exponentials, conditional softmax rescaling and tensor-memory techniques. Their paper reported up to 1.3× speedup over cuDNN 9.13 and 2.7× over Triton on B200 GPUs using BF16.[13][14]

The conference announcement also described Together Megakernel, a production implementation of open research developed with Stanford collaborators. For an unnamed voice-agent customer, Together’s kernels team selected a model architecture and optimized an entire model into one kernel. Together reported time to the first 64 tokens falling from 281 milliseconds on the customer’s previous B200 deployment to 77 milliseconds on its H100 deployment. The result compared deployments on different hardware; the announcement does not isolate the kernel implementation’s contribution.[13]

The remaining announcements extended optimization beyond individual model calls. ThunderAgent, open-sourced that day, scheduled whole agentic workflows to address repeated context computation, memory imbalance and tool-resource cleanup. The Reinforcement Learning API exposed inference and training as separately configurable layers while handling synchronization and scheduling. ATLAS-2 updated speculative decoders using accepted and rejected tokens from live traffic. Cache-aware prefill–decode disaggregation routed cold and warm requests to different tiers so reusable context could be fetched rather than recomputed. The announced together.compile integration into Dedicated Container Inference was still described as coming soon.[13]