Aditya Kusupati
email:

CV | Scholar | Twitter
(Old-ish) Research Statement

I am a Researcher at Google DeepMind.

My research interests lie in fundamental and practical machine learning working towards adaptive intelligence -- currently working on continual learning -- but my day-to-day nearly rivals the randomness marked by cosmic rays.

I got my PhD from the University of Washington jointly advised by Ali Farhadi and Sham Kakade while closely working with Prateek Jain at Google Research → DeepMind as a Student Researcher. Before joining PhD, I spent two amazing years as a Research Fellow at Microsoft Research with Manik Varma and Prateek Jain. In a past life, I earned a Bachelor's in CS with Honours and a Minor in EE from IIT Bombay where I worked with Soumen Chakrabarti.

  • 2018: Early deep learning models to work in < 16KB memory & on-device wakeword detection.
  • 2020: End-to-end differentiable/learnable sparsity.
  • 2021: End-to-end discrete bit-code learning → differentiable search → precursor to Semantic Ids & Generative Retreival.
  • 2022-26: Matryoshka learning paradigm for adapative intelligence.
    • 2022: Matryoshka Representation Learning (MRL) -- powering most deployed search systems.
    • 2023: Matryoshka Transformer (MatFormer) -- powering widely deployed large transformer models.
    • 2024-26: Matryoshka Quantization (MatQuant), adaptive tokenizers, better SAEs and 3D representations leveraging matryoshka learning.
  • 2024: Fundamentals of latent thinking through superposed decoding -- few months before thinking models and coconut paper.

Template: this.