Robert McCloy

Robert MacCloy is the CTO and cofounder of Scrunch, which he started with Chris Andrew to commercialize his work on how AI platforms and tools use web content. The two began building the platform in 2023 around the idea that an important website visitor would be an AI agent acting on a person’s behalf. MacCloy appeared in Latent Space’s July 23, 2025 episode AI is Eating Search, whose description spells his name Robert McCloy; his personal website and Scrunch biography use MacCloy.[3][2][1]

In his February 10, 2026 article What actually works for AI search optimization, MacCloy recommends serving clear, information-dense HTML directly from the server at existing URLs. His argument concerns search and retrieval, which he distinguishes from collecting training data. He emphasizes complete page text, text alternatives for rich media and minimal navigation clutter. Keeping existing URLs also means that citations lead human users back to the website’s normal pages.[4]

MacCloy’s proposed mechanism is to identify known AI user agents at the CDN or load balancer and route them to structured HTML while retaining the interactive website for people. He connects this approach to Scrunch’s Agent Experience Platform: an enterprise content-delivery product addressing the difficulty of serving both audiences. In the same article, he argues that most AI agents do not see JavaScript-rendered content and recommends checking raw HTTP logs to establish what they actually request.[4]

His guidance extends beyond whether a brand appears in an AI answer. He recommends inspecting low-level access logs, checking whether answers convey the intended message and whether citations point to useful pages, and testing the content an agent actually receives. Starting with branded questions, he argues, can expose accessibility and comprehension problems without the added variability of competitive ranking. His instruction is concise: “Be intentional, look at your data, test behaviors, and iterate.”[4]

Earlier, in LLMs are already good enough to revolutionize the internet, published November 11, 2024, MacCloy described spending most of his recent time writing generative-AI code for chatbots, data extraction, automation, moderation and classifiers. He acknowledged unreliable agents and copilots while arguing that better application integration and form factors could produce substantial improvements without stronger underlying models. “LLMs are powerful technology but not digital gods,” he wrote. His practical examples included defining a schema and using Instructor for structured extraction, and transforming content to suit a reader’s knowledge or preferences.[6]

In his July 9, 2025 essay The crawl is coming from inside the house, MacCloy argued that changing consumer preferences mattered more than a narrative focused solely on competition between technology platforms. He predicted less conventional browsing because much of it had become a chore, while preserving exceptions for fashion, art and reading for pleasure. He described AI browsers as another way to connect AI to the existing web, operating from consumer devices and residential networks rather than centralized infrastructure.[5]

MacCloy also supported publishers seeking compensation from AI labs that train on their material and platforms that remix it. “As a society, we need to decide how we value human writing and how to pay for it,” he wrote. At the same time, he argued that blocking platforms would not preserve publishers’ existing business value if consumers stopped browsing.[5]

Before his AI-content work, MacCloy built data systems and led software engineering organizations. His personal account dates his advertising-technology work to 2006–2010, notably at Quantcast, involving auction systems, automatic audience segmentation, anti-abuse and anti-fraud systems, and hyperscale data warehousing. From 2010 to 2021, he worked at Hearsay, progressing from an early employee to VP Engineering and CTO. Hearsay developed marketing and customer-experience software for investment and insurance institutions; this earlier work supplies the data-infrastructure and enterprise-software context for his subsequent role at Scrunch.[3][2]