“A testament to what an earnest, incredibly hardworking technical expert can accomplish when they commit to public service… he has materially influenced many of the bold efforts that the UK has been making to invest in frontier AI Safety.” – Ian Hogarth, Chair of UK AISI

“Few people deserve more credit than Nitarshan for the UK’s enormous strides in state capacity in AI over the last 18 months: an incredibly capable and dedicated civil servant.” – Matt Clifford, Chair of ARIA

“Nitarshan is the shadowy cofounder of UK AISI and all-round cool person”

“Nitarshan is a PhD student in Cambridge, previously a student at MILA, a top machine learning lab in Canada. He is interested in topics of AI safety among others. He also has an impressive repertoire of about five things he can say in Hungarian (and maybe even more).”

“Beware of Nitarshan in general… He asks thought-provoking questions that will make you rethink your whole research agenda”

“Quite fun and interesting. Those are his parameters.”

Bookshelf

Darwin among the Machines (1863)
Progress and Poverty (1879)
The Strenuous Life (1899)
Citizenship in a Republic (1910)
Dust of Snow (1923)
Economic Possibilities for our Grandchildren (1928)
The Human Use of Human Beings (1950)
Computing Machinery and Intelligence (1950)
Can We Survive Technology? (1955)
Man-Computer Symbiosis (1960)
All Watched Over by Machines of Loving Grace (1967)
Spring Snow (1969)
See How It Flies (1995)
Sidewinder: Creative Missile Development at China Lake (1999)
Why the Future Doesn’t Need Us (2000)
The Dream Machine (2001)
Strategic Computing: DARPA and the Quest for Machine Intelligence, 1983-1993 (2002)

Underpaid Labour

GitHub | Google Scholar | Semantic Scholar | OpenReview

Open Problems in Technical AI Governance
arXiv
Anka Reuel, Ben Bucknall, Stephen Casper, Tim Fist, Lisa Soder, Onni Aarne, Lewis Hammond, Lujain Ibrahim, Alan Chan, Peter Wills, Markus Anderljung, Ben Garfinkel, Lennart Heim, Andrew Trask, Gabriel Mukobi, Rylan Schaeffer, Mauricio Baker, Sara Hooker, Irene Solaiman, Alexandra Sasha Luccioni, Nitarshan Rajkumar, Nicolas Moës, Jeffrey Ladish, Neel Guha, Jessica Newman, Yoshua Bengio, Tobin South, Alex Pentland, Sanmi Koyejo, Mykel J Kochenderfer, Robert Trager

IDs for AI Systems
arXiv
Alan Chan, Noam Kolt, Peter Wills, Usman Anwar, Christian Schroeder de Witt, Nitarshan Rajkumar, Lewis Hammond, David Krueger, Lennart Heim, Markus Anderljung

Visibility into AI Agents
arXiv FAccT 2024
Alan Chan, Carson Ezell, Max Kaufmann, Kevin Wei, Lewis Hammond, Herbie Bradley, Emma Bluemke, Nitarshan Rajkumar, David Krueger, Noam Kolt, Lennart Heim, Markus Anderljung

Reclaiming the Digital Commons: A Public Data Trust for Training Data
arXiv AIES 2023 Blog
Alan Chan, Herbie Bradley, Nitarshan Rajkumar

Harms from Increasingly Agentic Algorithmic Systems
arXiv FAccT 2023
Too many authors tbh

Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamics
arXiv Code ICLR 2023 ICML 2022 Workshop (DataPerf)
Shoaib Ahmed Siddiqui, Nitarshan Rajkumar, Tegan Maharaj, David Krueger, Sara Hooker

Evaluating the Text-to-SQL Capabilities of Large Language Models
arXiv Code
Nitarshan Rajkumar, Raymond Li, Dzmitry Bahdanau

Myriad: A Real-World Testbed to Bridge Trajectory Optimization and Deep Learning
arXiv Code NeurIPS 2022
Nikolaus Howe, Simon Dufort-Labbé, Nitarshan Rajkumar, Pierre-Luc Bacon

Self-Supervision for Data Interpretability in Image Classification and Sample Efficiency in Reinforcement Learning
MSc Thesis
Nitarshan Rajkumar

Pretraining Representations for Data-Efficient Reinforcement Learning
arXiv Code NeurIPS 2021 ICLR 2021 Workshop (SSL-RL)
Max Schwarzer, Nitarshan Rajkumar, Michael Noukhovitch, Ankesh Anand, Laurent Charlin, Devon Hjelm, Phil Bachman, Aaron Courville

In Search of Robust Measures of Generalization
arXiv Code NeurIPS 2020
Karolina Dziugaite, Alex Drouin, Brady Neal, Nitarshan Rajkumar, Ethan Caballero, Linbo Wang, Ioannis Mitliagkas, Dan Roy

Unpaid Labour

Weight Uncertainty in Neural Networks