Ruslan Salakhutdinov

wiki
, , , ,
1

Canadian researcher of Tatar origin and UPMC Professor at Carnegie Mellon University, known for foundational contributions to deep belief networks, deep Boltzmann machines, Dropout, and Bayesian Program Learning, and currently Vice President of Research in Generative AI at Meta.

Profile
Born c. 1980, Tashkent, Uzbekistan (then USSR)
Nationality Canadian
Current Institutions Meta (VP Research, Generative AI); Carnegie Mellon University (UPMC Professor, Machine Learning Department)
Research Areas Deep Learning, Probabilistic Graphical Models, Large-Scale Optimization, Multimodal LLMs, AI Agents
Doctoral Advisor Geoffrey Hinton
Doctoral Thesis Learning Deep Generative Models (University of Toronto, 2009)
CMU Website cs.cmu.edu/~rsalakhu
Google Scholar Ruslan Salakhutdinov — cited 264,000+ times
X / Twitter @rsalakhu

Overview

Ruslan “Russ” Salakhutdinov is one of the central figures of the deep learning renaissance that began in the mid-2000s. Trained under Geoffrey Hinton at the University of Toronto, he co-authored the papers on deep belief networks and deep Boltzmann machines that helped re-establish neural networks as the dominant paradigm in machine learning. He later co-invented Dropout — arguably the most widely used regularisation technique in the history of deep learning — and contributed to Bayesian Program Learning, an influential model of human one-shot concept acquisition. His career has bridged academia and industry in an unusual way: he has held a full professorship at Carnegie Mellon University continuously since 2016 while also serving as the first Director of AI Research at Apple and, since 2024, as Vice President of Research in Generative AI at Meta. His publications have accumulated over 264,000 citations, making him one of the most cited researchers in the history of the field.

Early Life & Education

Salakhutdinov was born around 1980 in Tashkent (then Soviet Uzbekistan) and is of Tatar origin. He emigrated to Canada and pursued his graduate education entirely at the University of Toronto. He was considering quitting the field of artificial intelligence when he met Geoffrey Hinton in 2004, but changed his mind after Hinton invited him to take part in a project focused on a new way to train artificial neural networks, which Hinton dubbed “deep belief networks.” That meeting proved decisive: Salakhutdinov became one of Hinton’s closest collaborators through his doctoral years, co-authoring the papers that would launch the modern deep learning era.

He completed his PhD in machine learning (computer science) at the University of Toronto in 2009. His thesis, Learning Deep Generative Models, addressed unsupervised learning of hierarchical probabilistic models. After his PhD, he spent two post-doctoral years at MIT’s Department of Brain and Cognitive Sciences and the Computer Science and AI Laboratory (CSAIL), 2009–2011.

Career

University of Toronto (2011–2016)

Salakhutdinov joined the University of Toronto as an Assistant Professor in the Department of Computer Science and Department of Statistics in 2011. During this period he received several major early-career awards, co-invented Dropout, and developed influential work on one-shot learning with collaborators at MIT. He became a Fellow of the Canadian Institute for Advanced Research (CIFAR) and attracted funding from NSERC, the Sloan Foundation, Google, Microsoft, and Samsung.

Carnegie Mellon University (2016–present)

Salakhutdinov joined the Machine Learning Department at Carnegie Mellon as an associate professor in 2016. He was subsequently appointed UPMC Professor of Computer Science, an endowed chair funded by UPMC to advance work in AI, machine learning, and data analytics for healthcare. At CMU his research expanded into natural language processing and multimodal learning, and his group produced influential work on reading comprehension, graph neural networks, language-grounded agents, and multimodal foundation models. He has supervised numerous PhD students whose work has had broad impact across NLP, vision, and RL, including Zhilin Yang, co-first-author of XLNet.

Apple — Director of AI Research (2016–2020)

In 2015, Salakhutdinov co-founded Perceptual Machines, a company focusing on AI technologies, which was later acquired by Apple. Salakhutdinov joined Apple as its Director of AI Research in 2016 — Apple’s first person to hold that title — where he led key AI and machine learning research initiatives. He left in early 2020 to refocus on his CMU role.

Meta — VP Research, Generative AI (2024–present)

In June 2024, Salakhutdinov joined Meta as Vice President of Research in Generative AI, where his work focuses on multimodal large language models (LLMs) and AI agents. He retains his UPMC Professorship at CMU concurrently.

Key Contributions

  • Deep Belief Networks (DBNs) (2006) — Co-authored with Geoffrey Hinton the landmark paper demonstrating efficient greedy layer-wise pretraining of deep generative models. Published in Science and as a NIPS paper, this work is widely credited as one of the starting points of the modern deep learning renaissance, showing that deep networks could be trained effectively for the first time at scale.

  • Deep Boltzmann Machines (DBMs) — Introduced at AISTATS 2009 with Hinton, DBMs extended the undirected graphical model framework to multiple layers, offering a fully probabilistic deep architecture with approximate inference procedures. Subsequent work on efficient learning of DBMs (AISTATS 2010) provided practical training algorithms that enabled application to multimodal data.

  • Restricted Boltzmann Machines for Collaborative Filtering — Applied RBMs to the Netflix Prize recommendation problem, achieving competitive results and demonstrating that probabilistic latent variable models could scale to industrial recommendation tasks.

  • Dropout (JMLR 2014) — Co-authored with Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, and Ilya Sutskever. The paper proposed randomly dropping neural network units during training as a simple, highly effective regularisation technique. It became one of the most cited papers in machine learning history and remains a standard component of neural network training.

  • Bayesian Program Learning / One-Shot Concept Learning (Science, 2015) — Co-authored with Brenden Lake and Josh Tenenbaum. The paper proposed a probabilistic model of human handwritten character learning that could match or exceed human performance in recognising novel characters from a single example, establishing one-shot learning as a major research agenda and connecting cognitive science with deep learning.

  • XLNet (NeurIPS 2019) — Co-authored with Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, and Quoc V. Le. XLNet proposed a generalised autoregressive pretraining method combining the advantages of autoregressive language modelling and BERT-style bidirectional context, outperforming BERT on 20 tasks and achieving state-of-the-art results on 18 tasks at time of release, including SQuAD, GLUE, and RACE.

  • Multimodal Large Language Models (2023–) — At CMU and Meta, Salakhutdinov’s group developed early work on grounding language models to images for joint multimodal input and output (FROMAGe, GILL), and on multimodal learning without labelled multimodal data, contributing to the foundation of modern multimodal LLM research.

Awards & Recognition

  • Alfred P. Sloan Research Fellowship (2013–2015) — Awarded for early-career excellence in research.
  • Microsoft Research Faculty Fellowship (2013–2015) — Recognising outstanding junior faculty in computer science.
  • Early Researcher Award (2012–2017) — Ontario Ministry of Research and Innovation award for outstanding early-career research.
  • Connaught New Researcher Award (2012–2014) — University of Toronto recognition for research achievement.
  • Google Faculty Research Award (2014–2015) — Grant in recognition of research excellence.
  • UPMC Professorship, Carnegie Mellon University — Endowed chair in AI, machine learning, and health data analytics.
  • Nvidia Pioneers of AI Award — Recognising foundational contributions to the field.
  • Senior Fellow, CIFAR — Senior fellowship in the Learning in Machines and Brains programme of the Canadian Institute for Advanced Research.
  • Google Scholar citations — Over 264,000 total citations, placing him among the most cited AI researchers worldwide.

Key Relationships

  • Geoffrey Hinton — PhD supervisor at the University of Toronto and frequent co-author; Turing Award laureate (2018) and principal architect of the deep learning programme that Salakhutdinov helped advance. The chance meeting with Hinton in 2004 was a decisive turning point in Salakhutdinov’s career.
  • Josh Tenenbaum — MIT collaborator on Bayesian Program Learning and one-shot concept learning; their work bridging cognitive science and machine learning produced the Science 2015 paper on human-level concept learning.
  • Ilya Sutskever — Colleague at the University of Toronto and co-author on Dropout; later co-founder of OpenAI and Safe Superintelligence Inc.
  • Nitish Srivastava — PhD student and primary co-inventor of Dropout, whose JMLR 2014 paper became one of the most read papers in machine learning history.
  • Zhilin Yang — PhD student at CMU and first author of XLNet; one of Salakhutdinov’s most influential doctoral students.
  • Alex Krizhevsky — Co-author on Dropout; also Hinton doctoral student and creator of AlexNet, whose contributions shaped the same deep learning breakthrough era.
  • Eric Xing — CMU colleague in the Machine Learning Department; long-term collaborator on probabilistic models and NLP.

Personal Style

Salakhutdinov has consistently operated across the full spectrum from theoretical foundations to engineering implementation, a breadth unusual even among senior researchers. His early career contributions — DBNs, DBMs, Dropout — were not incremental refinements but paradigm-level interventions that reshaped how the field thought about unsupervised pretraining, regularisation, and generative models. In interviews he has emphasised the complementarity of academic freedom and industrial scale: he has noted that academia offers greater freedom to work on long-term problems, while industry research is exciting because it can impact millions of users when a core AI technology is developed. His CMU course materials and public tutorial lectures — including a four-part deep learning tutorial at the Simons Institute at Berkeley — have been widely used in graduate education globally.

References