Daniel Roy

wiki
, , , ,
1

Canadian mathematician and statistician who co-invented the Church probabilistic programming language, introduced the Mondrian process, proved fundamental limits on the computability of conditional probability, and built the PAC-Bayes generalization theory that connects Bayesian priors to deep learning — now at Google DeepMind.

Profile
Nationality Canadian
Current Institution(s) Google DeepMind (Visiting Researcher, London, 2026–present); University of Toronto (Professor, on leave — Dept. Statistical Sciences and Dept. Computer Science)
Research Areas Foundations of Machine Learning, Learning Theory, Bayesian Nonparametric Statistics, Probabilistic Programming, Computable Probability Theory, Online Learning, Nonstandard Analysis
Doctoral Advisor Leslie Kaelbling
Doctoral Thesis Computability, Inference and Modeling in Probabilistic Programming (MIT, 2011)
Website danroy.org
X / Twitter @roydanroy
GitHub droy
Google Scholar Daniel Roy

Overview

Daniel M. Roy (Dan) is a Canadian professor of statistics at the University of Toronto — on leave since January 2026 as a Visiting Researcher at Google DeepMind in London — who has produced foundational contributions across four distinct areas: probabilistic programming (as a co-creator of Church), Bayesian nonparametric statistics (the Mondrian process and graphex processes for random graphs), the mathematical logic of probability (computability of conditional probability, computable de Finetti measures), and modern learning theory (PAC-Bayes generalization bounds for deep networks, information-theoretic complexity of stochastic convex optimization). He received all three of his degrees from MIT, was a Royal Society Newton International Fellow and Emmanuel College Research Fellow at Cambridge, and joined the University of Toronto in the early 2010s, where he was a founding faculty member of the Vector Institute and subsequently served as its Research Director through December 2025. His ICML 2024 Best Paper, on information complexity in stochastic convex optimization, won recognition ten years after his most celebrated probabilistic programming work — a span that reflects the unusual breadth and duration of his foundational research program. He has trained a remarkable alumni network, with former students and postdocs now holding faculty positions at Chicago, Copenhagen, Waterloo, Oslo, Paris, and Imperial College London.

Early Life & Education

Roy was an early participant in the Research Science Institute (RSI) summer program in 1998. He completed all three of his degrees in Computer Science and Electrical Engineering at the Massachusetts Institute of Technology.

B.S. and M.Eng., MIT — His M.Eng. thesis, “Clustered Naive Bayes” (2006), was an early exploration of probabilistic modeling for classification. During his undergraduate and master’s years, he co-authored papers on program analysis and memory safety (OSDI 2004, ACSAC 2004, WODA 2004) with Martin Rinard’s group at MIT CSAIL — a systems background unusual for someone who would go on to specialize in probabilistic logic.

PhD, MIT CSAIL (2006–2011) — Roy completed his doctorate in MIT’s Computer Science and Artificial Intelligence Laboratory under Leslie Kaelbling, with deep involvement in Josh Tenenbaum’s Computational Cognitive Science group. His dissertation, Computability, Inference and Modeling in Probabilistic Programming, addressed the intersection of computability theory, mathematical logic, and Bayesian statistics. It won the MIT EECS George M. Sprowls Doctoral Dissertation Award — the department’s highest honor for doctoral theses. Key components of the dissertation include the Church probabilistic programming language, results on the computability of conditional probability, and an analysis of computable de Finetti measures.

Cambridge — Newton International Fellow and Emmanuel College Research Fellow — After his doctorate, Roy was awarded a Newton International Fellowship of the Royal Society and a Research Fellowship at Emmanuel College, Cambridge. At Cambridge he joined Zoubin Ghahramani’s Machine Learning Group and the Computational and Biological Learning Lab, deepening the probabilistic modeling foundations he had developed at MIT.

Career

MIT CSAIL — Doctoral Research and Church (2006–2011)

The most publicly recognized output of Roy’s doctoral years was the creation of Church, a probabilistic programming language introduced at UAI 2008 (with Noah Goodman, Vikash Mansinghka, Keith Bonawitz, and Josh Tenenbaum). Church extended Lisp-style computation with stochastic primitives, giving a principled way to specify complex generative models — including distributions over phrases in natural language, rendered images, and climate measurements — as programs, and performing Bayesian inference over their execution histories via Markov Chain Monte Carlo. Church became one of the founding documents of the probabilistic programming field and influenced a generation of systems including Anglican, Venture, Pyro, and Stan.

The deeper mathematical thread of his dissertation was a series of results on the computability of conditional probability and de Finetti measures. The central negative result — published in LICS 2011 and subsequently in the Journal of the ACM with Nathanael Ackerman and Cameron Freer — showed that conditional probability distributions are in general not computable from the joint, even when the joint is computable: Bayesian inference is, in a precise sense, algorithmically intractable. Companion results on computable de Finetti measures (Annals of Pure and Applied Logic, 2012, with Freer) extended these impossibility results to exchangeable sequences.

Cambridge — Newton International and Emmanuel College Fellow (2011–2013)

At Cambridge, Roy worked with Zoubin Ghahramani’s group on Bayesian nonparametric models and their probabilistic programming foundations. The Cambridge period produced, among other things, foundational survey work on Bayesian models of graphs and exchangeable random structures (IEEE PAMI, 2014, with Peter Orbanz), which established the graphex framework — a unified treatment of Bayesian models for relational and network data through the lens of exchangeability — as a structured research program.

University of Toronto — Professor (c. 2013–2026, with leave from 2026)

Roy joined the Department of Statistical Sciences at the University of Toronto and was cross-appointed in the Department of Computer Science and the UTSC Department of Computer and Mathematical Sciences. He is a CIFAR Canada AI Chair and was a founding faculty member of the Vector Institute, subsequently serving as its Research Director until December 31, 2025.

Mondrian processes and random forests. The Mondrian process (NIPS 2009, with Yee Whye Teh) is a stochastic process generating hierarchical rectangular partitions of multidimensional space, whose distribution is self-consistent across scales — a kind of spatial analogue of the Dirichlet process. Mondrian Forests (NIPS 2014, AISTATS 2015, 2016, with Balaji Lakshminarayanan and Yee Whye Teh) extended this to scalable online random forests with uncertainty quantification. The Mondrian Kernel (UAI 2016) gave an explicit kernel corresponding to the Mondrian process, connecting the partition model to kernel methods.

Graphex processes and sparse random graphs. A sustained research program with Victor Veitch and others generalized the Aldous-Hoover theorem for dense exchangeable graphs to the sparse regime via graphex processes — measures on edges of random graphs that generate sparse (sublinear density) random graph models consistent with observed network data. The main theoretical results appeared in the Annals of Statistics (2019) and contributed to the sparse exchangeable graph literature.

PAC-Bayes generalization bounds. Beginning with Gintare Karolina Dziugaite, Roy developed a line of work on PAC-Bayes bounds as practical, non-vacuous bounds on the generalization error of deep neural networks. “Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data” (UAI 2017) was the first demonstration of a non-trivial PAC-Bayes bound for a large neural network trained on real data. Follow-up papers — on data-dependent priors via differential privacy (NeurIPS 2018), the connection between entropy-SGD and PAC-Bayes (ICML 2018), and information-theoretic generalization bounds for SGD (NeurIPS 2019) — developed the theoretical and practical foundations of PAC-Bayes learning theory as applied to deep learning, and influenced the subsequent information-theoretic generalization literature.

Information complexity of learning. More recently, Roy and collaborators (including Mahdi Haghifam, Gintare Karolina Dziugaite, Idan Attias, and Roi Livni) developed a research program on information-theoretic complexity as a measure of the hardness of stochastic convex optimization and statistical learning. The ICML 2024 Best Paper, “Information Complexity of Stochastic Convex Optimization: Applications to Generalization and Memorization,” established tight information complexity bounds for SGD under standard convex optimization conditions — providing a unified framework explaining both the generalization properties and memorization phenomena observed in practice.

Nonstandard analytic foundations for decision theory. A third parallel line of work with Haosui Duanmu and David Schrittesser applies nonstandard analysis to classical decision theory: the results characterize admissibility of statistical procedures as equivalent to Bayes optimality relative to an infinitesimal prior, resolve long-standing foundational questions about the existence of regular conditional distributions under weak measurability assumptions, and provide nonstandard proofs of statistical minimax theorems.

Vector Institute Research Director (to December 2025). Roy served as Research Director of the Vector Institute, Canada’s national AI research institute, before stepping down on December 31, 2025 to join Google DeepMind as a Visiting Researcher.

Google DeepMind — Visiting Researcher (January 2026–present)

Roy joined Google DeepMind in London as a Visiting Researcher in January 2026 and is currently on leave from the University of Toronto.

Key Contributions

  • Church Probabilistic Programming Language (UAI 2008) — Co-created Church with Noah Goodman, Vikash Mansinghka, Keith Bonawitz, and Josh Tenenbaum. A Lisp-based language in which any computable distribution can be expressed as a probabilistic program and Bayesian inference performed via MCMC over execution histories. Founded the probabilistic programming field as a formal research area.

  • Mondrian Process (NIPS 2009) — With Yee Whye Teh. Introduced a stochastic process generating hierarchical rectangular partitions of ℝ^d that is self-consistent across scale — the canonical non-parametric prior for spatial partitioning. Basis for Mondrian forests and the Mondrian kernel.

  • Computability of Conditional Probability (LICS 2011; Journal of the ACM) — With Nathanael Ackerman and Cameron Freer. Proved that conditional probability distributions are not in general computable from the joint, even when the joint is computable. A fundamental impossibility result in the theory of Bayesian computation.

  • Computable de Finetti Measures (Annals of Pure and Applied Logic, 2012) — With Cameron Freer. Established computability results for de Finetti representations of exchangeable sequences, connecting the foundations of Bayesian nonparametrics to computability theory.

  • Graphex Processes and Sparse Exchangeable Graphs (Annals of Statistics, 2019; arXiv 2015) — With Victor Veitch. Generalized the Aldous-Hoover representation theorem to sparse random graphs via exchangeable random measures (graphexes), providing a principled nonparametric framework for statistical analysis of network data.

  • Non-Vacuous PAC-Bayes Bounds for Deep Networks (UAI 2017) — With Gintare Karolina Dziugaite. First demonstration of a tight, non-vacuous bound on generalization error for a deep neural network trained on real data, using an optimized PAC-Bayes prior. Initiated the modern information-theoretic generalization theory program for deep learning.

  • Information Complexity of Stochastic Convex Optimization (ICML 2024 Best Paper) — With Idan Attias, Gintare Karolina Dziugaite, Mahdi Haghifam, and Roi Livni. Established tight information complexity bounds for stochastic gradient descent under standard convex assumptions, providing a unified theory of generalization and memorization in learning algorithms.

  • Nonstandard Foundations of Decision Theory — With Haosui Duanmu and David Schrittesser. A series of results using nonstandard analysis to resolve foundational questions in statistical decision theory: characterizing admissibility as Bayes optimality with infinitesimal priors, proving existence of regular conditional distributions under weak assumptions, and giving nonstandard proofs of minimax theorems.

Awards & Recognition

  • MIT EECS George M. Sprowls Doctoral Dissertation Award (2011) — MIT’s highest honor for doctoral theses in computer science and electrical engineering.
  • Royal Society Newton International Fellowship — Competitive postdoctoral fellowship of the Royal Society of the UK, held at Cambridge.
  • Emmanuel College Research Fellowship, Cambridge — Junior research fellowship at one of Cambridge’s oldest colleges.
  • CIFAR Canada AI Chair — Designated as a Canada AI Chair by the Canadian Institute for Advanced Research.
  • Vector Institute Founding Faculty — One of the original academic founding members of the Vector Institute for Artificial Intelligence.
  • ICML 2024 Best Paper Award — For “Information Complexity of Stochastic Convex Optimization.”
  • Action Editor, JMLR — Serving on the editorial board of the Journal of Machine Learning Research.

Key Relationships

  • Leslie Kaelbling — MIT PhD advisor; a leading figure in robot learning and task planning; the mentorship relationship through which Roy entered the AI research community.
  • Josh Tenenbaum — MIT computational cognitive scientist and Bayesian modeling pioneer; intellectual home for the Church project; long-running co-author on probabilistic programming and cognitive science.
  • Vikash Mansinghka — MIT CSAIL researcher and probabilistic programming pioneer; co-created Church; continues to collaborate on probabilistic programming foundations.
  • Cameron Freer — Primary collaborator on the computability of conditional probability and de Finetti measures; the theoretical logic/computability thread of Roy’s career runs largely through this collaboration.
  • Yee Whye Teh — Oxford professor and Google DeepMind researcher; co-invented the Mondrian process; a recurring collaborator across nonparametric Bayes and random structures.
  • Gintare Karolina Dziugaite — The most prolific research collaborator of Roy’s Toronto years; co-led the PAC-Bayes program for deep networks; published over a dozen papers together and held postdoctoral and research positions in Roy’s group.
  • Zoubin Ghahramani — Cambridge ML Group head and postdoc host; provided the Bayesian nonparametric environment at Cambridge that connected Roy’s MIT foundations to British ML theory.
  • Peter Orbanz — Columbia statistician; co-authored the foundational graphex and exchangeable random structures survey; a key intellectual partner in the structural Bayesian nonparametrics program.
  • Victor Veitch — Roy’s most decorated doctoral student; co-developed graphex process theory; won the Statistical Society of Canada Pierre Robillard Award; now assistant professor at the University of Chicago.

Personal Style

Roy is a rare example of a theoretical computer scientist who has maintained a consistent mathematical standard across multiple research traditions — probabilistic programming, Bayesian nonparametrics, statistical logic, and learning theory — without reducing to a single technical toolkit. His work is characterized by an insistence on foundations: he is drawn to questions about when Bayesian inference is computable, when generalization bounds are truly tight, and when standard assumptions in statistics can be removed or weakened without sacrificing precision. The “removing assumptions from statistics” framing — which he used in a recent ETH Zürich talk — describes his current research program on nonstandard analytic foundations as accurately as it described his early work on computability impossibility. His students’ placement record is notable even by the standards of strong theory groups: Victor Veitch (University of Chicago), Jeffrey Negrea and Mufan Li (University of Waterloo), Jun Yang (University of Copenhagen), Yanbo Tang (Imperial College London), and others now occupy research faculty positions, and Haosui Duanmu holds a full professorship in China. He maintains a “marginalia” wiki on his site — a collection of errata, clarifications, and missed citations — that reflects his commitment to scientific integrity over presentation polish.

References