Google Brain

Google Brain was an AI research team within Google Research whose work included neural machine translation, TensorFlow, robotics, and generative models for music and art. It pursued both fundamental research and collaborations with Google’s product and infrastructure teams. On April 20, 2023, Google announced that Brain and DeepMind would combine into Google DeepMind.[3][2]

The April 2023 announcement brought together “the Brain team from Google Research, and DeepMind,” with Google’s computational resources backing the combined unit. Sundar Pichai said the consolidation would accelerate AI progress; that was the stated expectation behind the decision.[2]

The announced structure placed Demis Hassabis at the head of Google DeepMind. Jeff Dean would become Google’s Chief Scientist, reporting to Pichai and serving both Google Research and Google DeepMind. His remit included helping set research direction and leading strategic AI projects, beginning with a series of multimodal models. Google Research would continue its broader computer-science work under James Manyika.[2]

Before that consolidation, Brain had developed both learning methods and infrastructure for training and deploying them. Its earlier experiments and product collaborations show the range of work that preceded the merger.[4][3][2]

Latent Space’s January 2023 account dates Google Brain’s founding to 2011 and credits Jeff Dean and Andrew Ng. In a June 2012 announcement, Dean and Ng described experiments led by Quoc Le that distributed neural-network computation across 16,000 CPU cores, training models with more than one billion connections.[1][4]

The experiment addressed the labor required to label training data. Using still frames from unlabeled YouTube videos, a neuron learned to respond strongly to pictures of cats without receiving a single image labeled as a cat. The result illustrated learning a visual feature from unlabeled data; the researchers also explored using such data to supplement labeled examples for image classification.[4]

In late 2014, three Brain researchers published work on sequence-to-sequence learning with neural networks and demonstrated its use for machine translation. In 2016, Brain members worked with the Google Translate team to replace translation algorithms with an end-to-end learned system. By January 2017, that system was deployed in Google Translate for a growing number of language pairs.[3]

In his 2016 team retrospective, published January 12, 2017, Dean reported that the system closed the gap between the previous system and human-quality translations by up to 85% for some language pairs. The team also demonstrated zero-shot translation: translating between languages for which the system had never seen example sentence pairs.[3]

Brain open-sourced an initial version of TensorFlow in November 2015. Its API and reference implementation were released under Apache 2.0. The 2015 technical paper described an interface for expressing machine-learning algorithms and an implementation for executing them, spanning mobile devices through distributed systems with hundreds of machines and thousands of computational devices. Its extensive author list included Martín Abadi, Ashish Agarwal, Paul Barham, Greg Corrado, Jeffrey Dean, Rajat Monga, and Sherry Moore.[3][5]

TensorFlow’s 2016 systems paper explained its central mechanism: dataflow graphs represent computation, shared state, and operations that change that state. Graph nodes can be assigned across machines and across CPUs, GPUs, and Tensor Processing Units (TPUs). Representing shared state explicitly gives developers flexibility to experiment with optimization and training algorithms. The system supports both neural-network training and inference, and several Google services were already using it in production.[6]

Brain worked with Google’s compiler team on XLA, a TensorFlow backend compiler, and with the platforms team on specifications and high-level goals for the TPU custom machine-learning accelerator. Dean’s January 2017 retrospective reported more than 10,000 TensorFlow commits by over 570 contributors—a measure of the open-source project’s participation.[3]

In 2016, Brain collaborated with researchers at [X] on robotic arms that learned hand-eye coordination by pooling their experiences; Dean reported about 800,000 grasping attempts. That year the team also launched Magenta to explore art and machine intelligence, beginning with music and image generation. Its safety work included the white paper Concrete Problems in AI Safety, developed with researchers at Stanford, Berkeley, and OpenAI.[3]

Dean reported 27 Brain papers accepted at ICML, NIPS, and ICLR during 2016, alongside 34 submissions to the upcoming ICLR 2017. Brain’s first residency class arrived in June 2016: 27 residents selected from more than 2,200 applicants. The team also hosted 46 student interns during the year, mostly doctoral students.[3]

Google’s 2023 merger announcement included Transformers among the two predecessor teams’ collective accomplishments. First submitted on June 12, 2017, Attention Is All You Need introduced an architecture based on attention mechanisms, dispensing with recurrence and convolutions. The research was conducted by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin.[2][8][7]

The authors’ explanation used “bank” and “river” to illustrate self-attention. Each word’s representation is compared with the other words, producing attention scores that weight their contribution to an updated representation. This lets “bank” draw directly on “river” for context. The encoder repeats these updates in parallel across words; the decoder still generates output one word at a time, attending to earlier output and the encoder’s representations.[7]