LMArena

LMArena, renamed Arena on January 28, 2026, evaluates AI through real-world interactions and human feedback. Its public platform lets users compare model responses and vote on their preferences; its commercial AI Evaluations service provides labs, enterprises and developers with analysis of community feedback. Originating as an academic project at UC Berkeley, it expanded from chatbot comparisons into broader tasks and agent evaluation.[7][5][9]

The platform's evaluation approach began with Chatbot Arena in 2023. That earlier work established the anonymous comparisons and crowdsourced judgments on which LMArena built its public leaderboard.[3][7]

Chatbot Arena's May 3, 2023 announcement introduced anonymous, randomized model battles served through FastChat. Users chatted with two models side by side, then voted for the better response; model names appeared after the vote. The researchers used only votes cast while identities were hidden. The first published leaderboard ranked nine models using Elo ratings and approximately 4,700 votes collected over April 24–May 1, 2023.[3]

The organizers initially favored pairings they expected to be strong based on prior rankings, then switched to uniform sampling to improve overall ranking coverage. The launch announcement was authored by Lianmin Zheng, Ying Sheng, Wei-Lin Chiang, Hao Zhang, Joseph E. Gonzalez and Ion Stoica. It credited the Vicuna team for feedback, MBZUAI for donated compute, and Tianjun Zhang and Eric Wallace for discussions.[3]

The Chatbot Arena paper, submitted March 7, 2024, described crowdsourced pairwise comparisons and reported more than 240,000 votes. Its authors reported good agreement between crowd votes and expert raters. The paper brought together Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Hao Zhang, Banghua Zhu, Michael Jordan, Joseph E. Gonzalez and Ion Stoica.[4]

LMArena became an independent platform in September 2024 while continuing its partnership with LMSys. Its scope broadened to WebDev Arena, complex tasks and red-teaming. LMSys remained a broader research collective behind projects including Vicuna, Chatbot Arena, SGLang, S-LoRA and RouteLLM; that collective's portfolio was distinct from the independent LMArena platform.[2]

Latent Space's account of its NeurIPS 2025 interview with Angelopoulos credits Anjney Midha at a16z with incubating the academic project, forming an entity and providing grants before the team committed to starting a company. By the NeurIPS 2025 interview, the company had raised $100 million and was spending the funds on inference costs, a migration from Gradio to React, and hiring across machine learning, product and go-to-market, according to Latent Space's account. Community feedback also informed platform development: a March 2025 update described more than 20 members attending an alpha desktop-interface walkthrough and discussing the voting interface.[1][2]

On September 16, 2025, LMArena launched AI Evaluations, a commercial service for enterprises, model labs and developers. It offered in-depth evaluations based on community feedback, representative feedback samples for auditability, and service-level agreements committing to delivery timelines. At launch, LMArena reported more than 250 million cumulative conversations, more than 2 million monthly votes and more than 3 million monthly users.[5]

The commercial launch came with commitments to keep the public leaderboard free, evaluate models using the same methodology regardless of commercial interest, and maintain public access to key datasets and foundational research. LMArena also promised flexible pricing for open-weight models and nonprofit institutions, alongside continued sampling of publicly available and open-source models.[5]

Private testing became a contested part of Arena's evaluation model in 2025. The Leaderboard Illusion, first submitted April 29 and revised May 12, argued that providers able to test multiple private variants and selectively disclose results gained an advantage. Shivalika Singh and her coauthors also alleged that sampling differences between proprietary and open models created unequal access to evaluation data and encouraged overfitting to Arena-specific dynamics.[10]

In its May 9 response, Arena disputed the paper's treatment of open-weight models and said its reported data-driven performance gain concerned Arena-Hard—a static, 500-item benchmark using an LLM judge—rather than the live human-voted platform. Arena said stronger models were sampled more often to improve user experience regardless of provider. It also announced preliminary plans to clarify model retirement and mark retired models, explicitly describe access to multiple prerelease tests subject to capacity, and label scores “provisional” until 2,000 additional fresh post-release votes had accumulated when more than ten models were tested in parallel before release.[11]

On January 6, 2026, LMArena announced it had raised $150 million in Series A funding at a $1.7 billion post-money valuation. Felicis and UC Investments led the round, with participation from Andreessen Horowitz, The House Fund, LDVP, Kleiner Perkins, Lightspeed Venture Partners and Laude Ventures. The announcement identified Angelopoulos as co-founder and CEO and said the funds would support platform operations, expansion of the technical team and research.[6][8]

LMArena reported more than 5 million monthly users across 150 countries generating more than 60 million conversations a month. Its January financing announcement also reported that the annualized consumption run rate had surpassed $30 million in December 2025, less than four months after the evaluation product launched. That figure described an annualized rate of consumption, rather than revenue collected over a year.[6]

On January 28, LMArena changed its name to Arena. The organization explained that its expansion beyond language models meant it had “outgrown the ‘LM,’” and introduced the arena.ai domain. Alright Studio collaborated on the visual identity and brand strategy; Fern Studio supplied animation, with typefaces from Klim Type Foundry and Optimo.[7]

In a June 29 announcement updated July 8, Angelopoulos reported that Arena had crossed $100 million in annualized revenue run rate within eight months of launching its enterprise offering. He also reported more than 700 million total conversations, more than 82 million total votes and more than 10 million monthly visitors. The financial milestone was a run rate, rather than $100 million collected during those eight months.[8]

Agent Arena, announced June 4, 2026, introduced a different ranking method for agents performing multi-step work. Instead of pairwise votes, its causal tracing approach randomized component selection and estimated the improvement in observed outcomes relative to a baseline distribution. Although the framework accommodated multiple agent components, the described production leaderboard ranked only orchestrator models—the main models deciding which tools to call.[9]

Its initial five signals combined user judgments with environmental feedback: final approval or disapproval, praise versus complaint, acceptance of corrections, recovery from model-caused bash errors, and calls to nonexistent tools. Arena estimated effects with self-normalized importance weighting, reported 95% confidence intervals and applied time-decaying weights to emphasize recent observations. These operational signals measured different aspects of agent behavior rather than a single universal definition of correctness.[9]

The methodology report described a recent seven-day sample of 160,480 tasks across 128,244 sessions, with 75.6% of sessions using at least one tool. Tasks included coding, research, planning, document creation and multimodal work; observed artifacts included a sports-TV schedule site and a financial-research RAG pipeline. In the June scale announcement, Angelopoulos reported more than 5 million Agent Mode turns per month and 10% week-over-week growth.[9][8]

Arena also supported research into the software harness surrounding a coding model. HarnessTax, published September 16, 2026, compared 21 pairings of seven models with Claude Code, Codex CLI and Pi. It used 30 sampled tasks each from SWE-bench Lite and Terminal-Bench 2.0, three attempts per task, high effort settings and a 100-turn cap; costs used a fixed September 1 direct-API price list. The researchers reported similar success rates at substantially different costs across tested harnesses, while limiting the findings to two open benchmarks that models might have encountered during training.[12]

HarnessTax credited Melissa Z. Pan, S. Yang, Negar Arabzadeh, Wei-Lin Chiang, Ion Stoica and Matei Zaharia. Arena Intelligence sponsored API access for the profiling experiments, alongside AWS compute credits from the Amazon AI Fellowship and Anthropic API credits from Laude.[12]