Nyla Worker

Nyla Worker is an AI product manager whose work spans inference optimization, synthetic training data and conversational 3D characters. At eBay, she optimized computer-vision inference for image search; at NVIDIA, she worked with customers deploying edge AI and became a product manager for Omniverse Replicator. She announced her move to Convai as Director of Product Management in September 2023. By September 2024, she had joined Google.[1][2][3]

Worker argues that useful AI requires close attention to deployment: “if these things are truly useful, you should be obsessing about inference.” In her Latent Space interview, published September 3, 2024 and recorded during her Convai tenure, she distinguished maximizing throughput from meeting the response time a person needs. Increasing batch size can process more images, but the resulting latency must still fit the application.[1][4]

She described efficiency gains as a combination of better compute utilization, model optimization, cheaper hardware and smaller models that meet the same requirements. Her clarification of the interview treated these as distinct contributors. In the interview, she also cautioned that a later throughput benchmark was not necessarily comparable to her earlier production workload, which had a latency constraint.[4][1]

Those deployment judgments grew out of earlier computer-vision work. Worker recalled manually classifying astronomical images during college astrophysics research, then encountering Source Extractor, a system that used a neural network trained on synthetic data to distinguish stars from galaxies. Seeing a learned system automate classification prompted her to investigate how such models were trained and deployed. She moved into computer-vision research and later worked with GPUs and edge devices at eBay.[1]

At eBay, Worker performed software optimization on the hardware team, helping AI research models run efficiently on the available equipment. She recalled using TensorRT on a V100 GPU to optimize a ResNet-50 model whose embeddings supported image search. Her decision was to increase batching while preserving the product's seven-millisecond inference target: the production workload went from one image to four images within that interval. This increased throughput without requiring a shorter response time.[1]

Worker met NVIDIA through her eBay work and entered its solutions-architect rotational program, supporting customers using AI in retail, manufacturing, robotics and vehicles. She subsequently chose to join the Edge AI team. She described team work on INT8 quantization-aware training, which trains models to operate under reduced-precision constraints, and pruning. The acceptable tradeoff depended on the deployment: a license-plate detector required attention to accuracy and recall, while a manufacturing line could impose especially tight latency requirements.[1]

Worker recalled finding that customers' specialized data needs often exceeded what existing image datasets could supply. Changing retail packaging and scratches on manufactured cars were examples. She worked with Amazon on synthetically changing package tape in a 3D environment so vision models could be retrained for new tape. For robotic manipulation, she worked on generating object-pose data, avoiding the need to attach physical tags that altered the objects being observed. She credited collaborating researchers for work on 3D synthetic-data generation.[1]

In June 2022, Worker and Bhumin Pathak coauthored a technical explanation of Omniverse Replicator. They described a pipeline in which randomizers vary assets, materials, lighting and camera positions; annotators extract labeled outputs from rendering; and writers convert images and annotations into model-specific training formats. Their central qualification was that generating synthetic data is iterative: models must be tested on real datasets, and the simulated training distribution adjusted accordingly.[2]

Worker's emphasis on deployment also shaped her evaluation practices. “I don't trust benchmarks, especially when the numbers are close,” she said, recalling benchmark gaming in computer vision. With customers, she instead asked whether a test set represented the actual environment and the behavior they needed to validate.[1]

In her September 4, 2023 announcement, Worker described leaving NVIDIA after four and a half years to pursue productizing AI at Convai. She wanted non-player characters that could understand and respond contextually, and saw potential implications for human-computer interaction and, prospectively, human-robot interaction. In the later interview, she connected the move to the possibility of combining generated 3D worlds with embodied characters that converse and plan tasks.[3][1]

Worker described Convai's team combining language models and retrieval with speech recognition, speech synthesis, facial animation, perception and an action model. Her product judgment was that conversation alone was insufficient: characters needed to respond naturally and act within their surroundings. In a ramen-shop demonstration developed with NVIDIA, picking up an object while conversing made response time and action timing part of the same experience. She also described Convai characters helping users learn activities and meet people in Second Life.[1]

Worker also experimented with generated entertainment. She recalled making an AI-generated podcast with her roommate around ChatGPT's release, aiming to generate the title, imagery, voice and editing rather than supply those elements manually. Although she judged the resulting podcast poor, the experiment helped her explore a broader possibility: AI-generated characters and content as a new form of entertainment.[1]