Barret Zoph

wiki
, , , ,
1

Research scientist who invented Neural Architecture Search, co-created the Switch Transformer, and architected OpenAI’s post-training function before a turbulent spell as CTO of Thinking Machines Lab.

Profile
Nationality American
Current Institution(s) OpenAI (rejoined January 2026)
Research Areas Neural Architecture Search, AutoML, Sparse Mixture-of-Experts, Post-Training, RLHF, Computer Vision, NLP
Education Computer Science background; affiliated with USC Information Sciences Institute (NLP group, 2014–2016) — higher degree details unconfirmed
Website barretzoph.github.io
X / Twitter @barret_zoph
GitHub barretzoph
Google Scholar Barret Zoph

Overview

Barret Zoph is an American AI researcher whose career covers two of the most consequential methodological shifts in modern deep learning. At Google Brain he introduced Neural Architecture Search (NAS) with Quoc V. Le — the first demonstration that reinforcement learning could automatically discover competitive neural network designs — and later co-created the Switch Transformer, which made sparse Mixture-of-Experts at trillion-parameter scale practical. At OpenAI, where he joined just before the ChatGPT launch, he built the post-training team from scratch alongside John Schulman, overseeing the RLHF alignment and fine-tuning work that shaped GPT-4 and subsequent models. He then co-founded Thinking Machines Lab with Mira Murati in October 2024 as its CTO, but was dismissed in January 2026 amid disputed circumstances; Murati announced his departure citing “unethical conduct,” while Zoph publicly denied the characterisation and stated he was fired only after management learned he intended to leave. He returned to OpenAI the same day.

Early Life & Education

Little is publicly documented about Zoph’s early background or undergraduate education. His earliest published work appears under the affiliation of the Information Sciences Institute (ISI), a research unit of the University of Southern California (USC), where he worked with professors Kevin Knight and Daniel Marcu on statistical machine translation. His ISI papers, spanning 2015–2016, cover multi-source neural translation, vocabulary scaling for RNNs, and translator information theory. His transition to Google Brain in 2016 was direct and coincided with his first NAS submission. Whether he completed a graduate degree at USC or departed before finishing is unconfirmed in available public sources.

Career

USC Information Sciences Institute — NLP Group (c. 2014–early 2016)

Working within Kevin Knight’s NLP group, Zoph co-authored several papers on neural machine translation, including multi-source architectures (NAACL 2016, oral presentation), transfer learning for low-resource MT (EMNLP 2016), and vocabulary efficiency for large-scale RNNs. This early work established his fluency with sequence-to-sequence neural models and exposed him to the challenge of hand-designing architectures for NLP — a frustration he would soon address algorithmically.

Google Brain (2016–August 2022)

Zoph joined Google Brain in early 2016 as a Research Scientist, rising to Staff Research Scientist. Over six years he produced a body of work that falls into three overlapping threads:

Neural Architecture Search and AutoML. The foundational paper, “Neural Architecture Search with Reinforcement Learning” (ICLR 2017, with Quoc V. Le), used a controller RNN trained with REINFORCE to generate child network descriptions and optimise them on CIFAR-10 and Penn Treebank. It achieved competitive accuracy against hand-designed networks, attracted a NYT feature, and launched the NAS sub-field. The follow-up NASNet (CVPR 2018, with Vasudevan, Shlens, and Le) showed that cell-level architectures found on CIFAR-10 transfer to ImageNet, achieving state-of-the-art with the design fully automated. ENAS (ICML 2018, with Pham, Guan, Le, and Dean) dramatically reduced the search cost via parameter sharing. These papers collectively catalysed Google’s AutoML product line and made NAS a mainstream topic.

Data augmentation and computer vision. Together with Ekin Doğuş Çubuk (later Periodic Labs co-founder), Jonathon Shlens, and Quoc Le, Zoph drove a sequence of principled augmentation papers: AutoAugment (CVPR 2019, oral), which searched for per-dataset augmentation policies via RL; RandAugment (NeurIPS 2020), which simplified the search to two hyperparameters and achieved better or equal accuracy; and learning-based augmentation for object detection (ECCV 2020). SpecAugment (Interspeech 2019), applying frequency and time masking directly to log-Mel spectrograms for ASR, became one of the most-cited speech augmentation techniques. AugMix (ICLR 2019) improved out-of-distribution robustness. Revisiting ResNets (NeurIPS 2021, spotlight) disentangled architecture improvements from training and scaling improvements, showing that much of the reported progress in classification had been driven by the latter.

Sparse language models. Zoph’s final Google Brain period focused on scaling language models through sparsity. Switch Transformers (JMLR 2022, equal contribution with William Fedus and Noam Shazeer) demonstrated single-expert routing at trillion-parameter scale with 4–7× pre-training speedup over dense T5 baselines. GLaM (2021, with Nan Du, Yanping Huang et al.) trained a 1.2-trillion-parameter MoE model that matched or exceeded GPT-3 on 29 NLP tasks at one-third the energy cost. ST-MoE (arXiv 2022, with Irwan Bello, Sameer Kumar et al.) addressed training instabilities in large sparse encoder-decoder models and achieved state-of-the-art on SuperGLUE. He also served as research tech lead for MuM (Multitask Unified Model), Google Search’s large multitask language model.

OpenAI — VP Research, Post-Training (September 2022–October 2024)

Zoph joined OpenAI in September 2022, arriving just weeks before the internal launch of ChatGPT. Along with John Schulman he built the post-training team from the ground up, overseeing alignment, RLHF pipelines, tool use, evaluations, multi-modality, and search integration. His team trained and shipped the models that powered ChatGPT and the API through this period, including contributions to GPT-4 and the infrastructure underlying the o1 reasoning model. He left in October 2024 to co-found Thinking Machines Lab.

Thinking Machines Lab — CTO & Co-Founder (October 2024–January 14, 2026)

Zoph joined Mira Murati’s newly founded AI lab as co-founder and CTO. The company raised a reported $2 billion seed round led by Andreessen Horowitz, with AMD, Nvidia, and Jane Street among the investors, and reached a valuation of approximately $12 billion. His tenure lasted roughly three months before ending in disputed circumstances. On January 14, 2026, Murati announced his departure at a company all-hands, with sources citing “unethical conduct”; a person familiar with the matter told Wired that he had been accused of sharing confidential company information with rival firms. Zoph categorically denied this characterisation to the Wall Street Journal: “Thinking Machines Lab terminated my employment only after it learned I would be leaving the company. Full stop. At no time did TML cite to me any performance reasons or any unethical conduct on my part as the reason for my termination and any suggestion otherwise is false and defamatory.” The WSJ also reported that Murati had concerns about his performance following her discovery, the previous summer, that he had been in a relationship with a junior colleague that had begun while both were at OpenAI — a relationship Zoph initially denied before he and the woman later confirmed it to Murati. The two accounts remain publicly unresolved. Soumith Chintala (PyTorch co-founder) was appointed CTO in Zoph’s place.

OpenAI (January 15, 2026–present)

Zoph returned to OpenAI on January 15, 2026, the day after his Thinking Machines dismissal, alongside Luke Metz and Samuel Schoenholz. Fidji Simo, OpenAI’s CEO of Applications, announced the rehires noting that OpenAI did not share Murati’s characterisation of Zoph’s conduct. His current role and responsibilities have not been publicly specified.

Key Contributions

  • Neural Architecture Search with Reinforcement Learning (ICLR 2017, with Quoc Le) — The paper that launched the NAS field; demonstrated RL-guided automated network design competitive with hand-crafted architectures on CIFAR-10 and Penn Treebank. Covered by NYT and the MIT Technology Review; Wikipedia credits this paper as the impetus for the NAS article.
  • NASNet / Learning Transferable Architectures (CVPR 2018, with Vasudevan, Shlens, Le) — Extended NAS to produce cell-level modules that transfer from CIFAR-10 to ImageNet, establishing the transferability paradigm for automated design.
  • AutoAugment (CVPR 2019, oral, with Cubuk, Mane, Vasudevan, Le) — First principled method for learning per-dataset image augmentation policies via RL; achieved SOTA on CIFAR-10/100, SVHN, and ImageNet.
  • SpecAugment (Interspeech 2019, with Park, Chan, Zhang, Chiu, Cubuk, Le) — Time and frequency masking augmentation applied directly to log-Mel spectrograms; became a standard component of ASR training pipelines.
  • Switch Transformers (JMLR 2022, equal contribution with Fedus and Shazeer) — Simplified MoE routing to a single expert per token, trained the first trillion-parameter language model, and achieved 4–7× pre-training speedups; a cornerstone of all subsequent sparse LLM families.
  • GLaM (arXiv 2021, with Du, Huang, Dai et al.) — 1.2-trillion-parameter MoE model requiring one-third the energy of GPT-3 to train, demonstrating the efficiency ceiling of sparsely-activated scaling.
  • Post-Training infrastructure at OpenAI — Co-built the RLHF and instruction-tuning pipeline with John Schulman that transformed GPT-4 into ChatGPT; led the team that shipped alignment, tool use, and multi-modal capabilities across the OpenAI API.

Awards & Recognition

  • ICLR 2017 Oral — NAS paper accepted as an oral presentation, the most competitive designation at the venue.
  • NeurIPS 2021 Oral — “Rethinking Pre-training and Self-training.”
  • NeurIPS 2021 and CVPR 2021 Spotlight presentations — For Revisiting ResNets and Progressive NAS follow-up work respectively.
  • NYT and MIT Technology Review features (2017) — The original NAS paper was featured in a New York Times article on AutoML and a Technology Review piece on Google’s use of AI to design AI.

Key Relationships

  • Quoc V. Le — Principal collaborator and mentor at Google Brain; co-authored the foundational NAS paper and NASNet; Le’s influence shaped Zoph’s research taste across both architecture search and large-scale pre-training.
  • William (Liam) Fedus — Equal co-author on Switch Transformers at Google Brain; the two later worked in parallel at OpenAI (Fedus as VP of Post-Training after Zoph) before Fedus departed to found Periodic Labs.
  • Noam Shazeer — Third co-author of Switch Transformers; Shazeer is the original inventor of the Mixture-of-Experts layer (2017) and now CEO of Character.AI.
  • Ekin Doğuş Çubuk — Recurring Google Brain collaborator on AutoAugment, RandAugment, SpecAugment, and adversarial examples; now co-founder of Periodic Labs with Fedus.
  • John Schulman — Co-built OpenAI’s post-training team with Zoph; Schulman later left OpenAI for Anthropic in 2024, one of several senior departures from the post-training function.
  • Mira Murati — CEO and former OpenAI CTO who recruited Zoph as CTO of Thinking Machines Lab; their professional relationship ended with a public and disputed dismissal in January 2026.
  • Kevin Knight — Early NLP mentor at USC Information Sciences Institute; Knight’s statistical MT group gave Zoph his grounding in sequence-to-sequence modelling before the deep learning era.

Personal Style

Zoph’s research output is characterised by a bias toward making things simpler and more scalable rather than more complex: NAS replaced hand-crafted heuristics with search; AutoAugment replaced intuition-based augmentation with automated policy search; Switch Transformers replaced complex multi-expert aggregation with a single routing step. He has been consistently willing to move between domains — NLP, computer vision, speech, sparse language models, post-training — following wherever the hardest unsolved scaling problems live. His public profile is relatively low for someone of his citation impact; he has rarely given long-form interviews and maintains a sparse online presence. The events of January 2026 thrust him into an unusually public controversy whose full account remains contested, with the two parties offering materially different characterisations.

References