Amir Haghighat

Amir Haghighat is the CTO and cofounder of Baseten, which started in 2019. His work centers on the infrastructure between a working machine-learning model and a dependable production service: model serving, runtime optimization, scaling, and the tools engineers need to use them.[2][3]

Haghighat described Baseten's changing workloads in a September 2025 interview. Early customers deployed regressors and classifiers for internal tasks such as fraud operations and content moderation. As customers adopted transformers and diffusion models, inference moved into the end user's path. Low latency, high throughput, and availability became more consequential, leading the founders to deepen both model and runtime optimization and infrastructure spanning regions and clouds.[3]

On Latent Space in January 2025, Haghighat and model-performance engineer Yineng Zhang jointly challenged a framework-centered account of inference. They framed production inference around three necessary layers: model performance, horizontal scaling, and developer experience for multi-step, multi-model workloads. A faster model still needs more replicas when traffic exceeds its capacity; regional GPU limits can then require those replicas to span regions and clouds.[1]

Haghighat and Rachel Rapp coauthored an explanation of multi-cloud capacity management, published in June 2025 and updated in July 2026. It describes a common layer for provisioning and managing compute across cloud providers. The same infrastructure supports managed, self-hosted, and hybrid deployments. In the hybrid case, customers can keep core workloads in their own cloud and dynamically route additional demand to Baseten Cloud, addressing traffic bursts without provisioning all peak capacity in advance.[4]

His technical writing also covers workload-specific trade-offs. With Tri Dao, Abu Qader, Bryce Dubayah, and Philip Kiely, he coauthored an August 2025 account of optimizing GPT OSS 120B. The team tested TensorRT-LLM, vLLM, and SGLang, fixed compatibility bugs, and checked both speed and correctness. For that deployment, they reported that tensor parallelism offered better latency while expert parallelism offered better system throughput; they chose tensor parallelism because latency was the priority.[5]

The same article explains speculative decoding: a smaller draft model proposes future tokens, and the target model validates them. The authors presented adding it as a next step, illustrating how inference optimization continues after an initial working deployment.[5]

Haghighat sees growth as a reason to preserve experimentation. In the September 2025 interview, he warned that product-market fit could disappear within nine months and emphasized “building fast, learning fast, and killing things when you need to kill them.” He described the leadership challenge as deciding how much to expand what already works while retaining the exploration that produced it.[3]

His developer-experience ambition was to let engineers move from discovering an open model to benchmarking, fine-tuning, and optimizing it without first mastering speculative decoding, attention optimizations, KV caches, or multi-cloud strategies. He wanted the platform to show engineers which tools to use and let them see their effects. In that 2025 account, he explicitly said the building blocks were not yet fully connected into the experience he envisioned.[3]

The founders' emphasis on operationalizing models predates generative AI. Haghighat traced their original inference decision to earlier work at Gumroad; his subsequent work at Clover Health supplies another example of the gap between model development and usable applications.[3][6]

Tuhin Srivastava recalled meeting Haghighat at Gumroad in 2012 and characterized him as its “de facto CTO” in a 2022 Greylock interview. Haghighat later explained that Srivastava and Phil Howes trained a fraud classifier whose difficult part was getting it to run quickly and reliably at production scale. That experience informed the founders' choice to focus Baseten on inference. He qualified the retrospective success: “We had the right thesis. I don’t know if the original reason was correct but it became correct over time.” Pretrained open models subsequently made inference many developers' starting point.[6][3]

At Clover Health, Haghighat led teams developing machine-learning models for population health management. Srivastava recalled one clinical workflow in which model outputs were placed in a spreadsheet for doctors to consult because engineering resources were constrained. He said the workflow imposed too much work on doctors and the effort was scrapped—a concrete example of how delivering predictions into everyday use could remain difficult even after a model had been developed.[6]