Sharif Shameem

Sharif Shameem founded Lexica, an AI image search and generation tool, and trained diffusion models there. He previously built Debuild, a GPT-3-powered code generation tool, and Vectordash, a GPU cloud backed by Y Combinator. His contributions include making generated code immediately visible in a browser, adding semantic image search to Lexica, and changing its model-training approach in response to users’ preferences.[6][1]

While working on Debuild in 2022, Shameem began building Lexica to make images and prompts from the Stable Diffusion Discord easier to explore. In his May 2023 interview, he recalled finding Discord’s search frustrating, collecting roughly 10 million images, building the website in about two days, and releasing it about 24 hours after Stable Diffusion’s weights became public. He initially expected a utility for dedicated Stable Diffusion users rather than a company.[1]

Shameem said he had set roughly one million users as the threshold for considering Lexica his full-time focus. By his account, it reached that threshold in its first week and a half, prompting the company to shift its attention from Debuild to Lexica. He reported subsequently raising about $5 million from Daniel Gross and separately participating in AI Grant.[1]

Lexica initially used PostgreSQL full-text search over the words in image prompts. Inspired by the visual search site Same Energy, Shameem then indexed CLIP image embeddings so searches could match image content rather than only prompt text. He used FAISS for nearest-neighbor retrieval because the embedding collection was too large for the regular database approach he had considered. In the architecture he described in May 2023, GPUs computed the embeddings while CPUs served the searches; the index also enabled similar-image search from an uploaded image.[1]

Shameem added generation directly to Lexica after watching users—and finding himself—switch between its gallery and separate Stable Diffusion interfaces. They would copy a prompt, paste it into another tool, and change a few words. Bringing an editor and generation into Lexica removed those repeated transfers.[1]

For model training, Shameem emphasized data quality and filtering. He described using preferences expressed on Lexica to infer pairwise image rankings, calculate Elo-style scores, and train aesthetic models that helped filter larger datasets. This connected the image-search product to the selection of training data.[1]

That approach changed with Aperture, Lexica’s image model. Shameem described V1 as an attempt at photorealism, but said some users preferred Stable Diffusion 1.5 because they wanted artistic and unusual abstract images. For V2, the team used art images from Lexica to train aesthetic ranking models, then applied those models to filter larger training sets. In May 2023, he described V3 as a further iteration with substantially more data. He distinguished versions based on different base models from fine-tuning updates released without a version increment.[1]

Before Lexica, Shameem’s work on Vectordash and Debuild involved renting GPUs for model training and turning language-model output into working interfaces. Those earlier projects provide context for his later search, generation, and agent work.[1][6]

Shameem studied computer science at the University of Maryland but dropped out. In the May 2023 interview, he corrected the host’s suggestion that he had graduated and described himself as still on indefinite leave. He recalled that a side project occupied much of his time and that an investor’s account of returning to school after a startup made leaving seem like a reversible decision.[1]

The GPU marketplace idea began with an exchange with his friend Alex, who was mining Ethereum on an NVIDIA 1080 Ti. Shameem wanted to train a character-level recurrent neural network on his iMessage history and paid to borrow Alex’s GPU. He recalled that the model was poor, but the rental worked for both of them: he obtained computing capacity and Alex earned more than he was making from mining. Shameem then pursued a marketplace connecting cryptocurrency miners’ GPUs with machine-learning researchers.[1]

Shameem attributed Vectordash’s commercial difficulties to its customer mix: students and researchers had limited budgets, startups already had cloud credits, and larger companies were reluctant to trust it with their data. The company pivoted to streaming video games from rented GPUs to users’ computers. He described hosts’ upload bandwidth as the largest constraint: a host could have many GPUs but insufficient network capacity to stream games from them simultaneously.[1]

In 2020, Shameem requested GPT-3 access on Twitter; he recalled Greg Brockman responding within hours. After discovering that GPT-3 could complete JavaScript functions, he began testing generated code by moving it from the playground into his editor and reloading a React development page. His Debuild prototype combined those steps in one browser page: a user entered a prompt, the API generated code, and the page compiled and displayed basic JSX React components. Shameem argued that showing the rendered result made the capability easier for an end user to understand.[1]

At Debuild, Shameem also explored fine-tuning smaller open-source models on examples collected through the product. In May 2023, he said the team had tried GPT-J and found FLAN-T5 the best fit for that particular use case, provided the prompts were short enough.[1]

Shameem’s gpt_calc.py, whose gist record displays a July 11, 2021 creation date, delegates answering a question to generated Python. It prompts OpenAI’s davinci engine to complete an answer_question function, constructs that function from the returned text, executes it with exec, and calls it to obtain the answer. A July 2023 Latent Space essay attributed inspiration for implementations by Shameem and Replit’s Amjad Masad to Riley Goodside’s earlier “Code Core” experiment.[9][2]

Shameem also demonstrated GPT-3 controlling Chrome for an AirPods-buying task in 2021. He later explained that the browser prototype grew from Debuild’s need to read API documentation and potentially register accounts or create API keys for users. It serialized webpages into a compact representation for GPT-3’s 2,048-token context, split longer pages into chunks, used subprompts to select actions, and executed the selected action in a loop. He described the prototype as prohibitively expensive and constrained by the small context window.[5][1]

In May 2023, Shameem identified two central browser-agent engineering problems: representing a webpage usefully to a text model and specifying actions unambiguously. A page with many identical “delete” buttons, for example, requires more than matching button text. He proposed combining terminal-browser representations with Vimium-style labels on interactive elements. For uncertainty, he suggested either training on browsing sessions amended to request help at failure points or explicitly prompting a model to ask the user when it did not know what to do next.[1]

On his Future page, headed August 2022, Shameem advocated collaborative language-model tools that combine models’ generation abilities with humans’ more refined taste. He envisioned assistants able to browse the web, generate code, and execute it to complete requests. In the May 2023 interview, he argued that AI capabilities exceeded the supply of useful products and urged builders to focus on compelling applications and adoption rather than primarily pursuing benchmark results.[8][1]

His Startup Operator’s Manual, dated December 22, 2019, advises: “Build the absolute smallest thing that can be considered a useful application and ship it.” It recommends two-day shipping cycles and flexibility when an idea needs to change. Shameem later described the list as reminders drawn from his mistakes, books, and Paul Graham essays. Looking back on Lexica, he argued that releasing its small initial version alongside Stable Diffusion mattered more than spending months adding features.[7][1]