Meta tags:

Headings (most frequently used words):

recent, about, me, selected, work, updates, druv, pai,

Text of the page (most frequently used words):
and (23), paper (14), 2024 (14), our (13), accepted (11), 2025 (11), was (10), transformers (10), via (10), white (9), box (9), rate (9), learning (9), deep (8), 2023 (7), transformer (6), reduction (6), with (6), for (6), attention (6), berkeley (5), coding (5), work (5), research (5), token (5), code (5), druv (4), pai (4), september (4), like (4), principles (4), neurips (4), sparse (4), diffusion (4), iclr (4), may (4), new (4), time (4), data (4), scale (4), eecs (3), interpretable (3), efficient (3), cpal (3), icml (3), theory (3), active (3), dormant (3), heads (3), mechanistically (3), demystifying (3), extreme (3), phenomena (3), llms (3), simplifying (3), dino (3), regularization (3), representations (3), distributions (3), about (3), email (3), student (3), google (3), prof (3), august (2), started (2), from (2), first (2), workshop (2), emergence (2), segmentation (2), minimalistic (2), papers (2), were (2), masked (2), completion (2), structured (2), january (2), denoising (2), models (2), representation (2), performant (2), jmlr (2), compression (2), all (2), there (2), nexusflow (2), scaling (2), spotlight (2), statistics (2), linear (2), variational (2), open (2), source (2), textbook (2), recent (2), the (2), github (2), chat (2), please (2), your (2), have (2), undergraduate (2), masters (2), note (2), methods (2), principled (2), experiments (2), how (2), developing (2), large (2), academia (2), 2026, powered, fork, minimal, mistakes, academicpages, jekyll, program, proposing, parameter, architecture, derived, october, xaia, november, closed, loop, transcription, convolutional, which, develops, connection, between, iterative, networks, uses, construct, autoencoder, global, geometric, analysis, maximal, comprehensive, reviewing, line, deriving, architectures, information, signal, processing, summer, scientist, internship, vision, oral, m3l, february, non, archival, track, only, unrolled, subspace, released, gave, tutorial, iaiss, updates, parts, nips, edge, memorization, website, selected, happy, graduate, school, etc, send, can, out, include, advising, title, currently, wrapping, ongoing, projects, making, transition, industry, not, additional, bandwidth, mentor, collaborators, over, extended, period, thank, you, understanding, notes, students, interests, broadly, lie, develop, scientific, mathematical, apply, these, analyze, simplify, improve, existing, build, approaches, such, tends, combination, controlled, larger, particularly, interested, structure, high, dimensional, including, environmental, feedback, interacts, this, impacts, generalization, laws, methodology, member, technical, staff, fortunate, advised, affiliated, supported, college, engineering, fellowship, prior, phd, completed, also, outside, previously, worked, recently, acquired, nvidia, deepmind, bair, jason, lee, jiantao, jiao, thinking, machines, lab, scholar, twitter, bay, area, usa, follow, empirical, teaching, coursework, publications,

Text of the page (random words):
about me druv pai druv pai publications coursework teaching cv druv pai ph d student uc berkeley developing theory for large scale empirical deep learning follow bay area ca usa email twitter github google scholar about me hi i m a member of technical staff at thinking machines lab and ph d student in eecs at uc berkeley in academia i m fortunate to be advised by prof yi ma prof jiantao jiao and prof jason lee i m affiliated with bair and supported by a uc berkeley college of engineering fellowship prior to my phd i completed a ba in cs and ms in eecs also at uc berkeley outside of academia i ve previously worked at google deepmind google research and nexusflow recently acquired by nvidia my research interests broadly lie in developing principled methodology for large scale deep learning i work to develop scientific and mathematical principles for deep learning apply these principles to analyze simplify and improve existing methods and build and scale new principled approaches as such my work tends to have a combination of theory controlled experiments and larger scale experiments i m particularly interested in how the structure of high dimensional data including environmental feedback interacts with deep learning methods and how this impacts representation learning generalization and scaling laws notes for undergraduate and masters students note 1 i m happy to chat about research graduate school etc please send me an email and we can work out a time please include advising chat in your email title note 2 i am currently wrapping up my ongoing research projects and making a transition to industry so i do not have additional bandwidth to mentor new undergraduate or masters student collaborators over an extended period of time thank you for your understanding selected recent work learning deep representations of data distributions open source textbook website github token statistics transformer linear time attention via variational rate reduction iclr 2025 spotlight paper code on the edge of memorization in diffusion models paper code white box transformers via sparse rate reduction compression is all there is jmlr 2024 parts at nips 2023 iclr 2024 cpal 2024 paper code simplifying dino via coding rate regularization icml 2025 paper code active dormant attention heads mechanistically demystifying extreme token phenomena in llms paper code recent updates september 2025 we gave a tutorial on learning deep representations of data distributions at iaiss 2025 august 2025 our new open source textbook learning deep representations of data distributions was released may 2025 our paper simplifying dino by coding rate regularization was accepted to icml 2025 february 2025 our papers simplifying dino by coding rate regularization active dormant attention heads mechanistically demystifying extreme token phenomena in llms and attention only transformers via unrolled subspace denoising were accepted to cpal 2025 non archival track january 2025 our paper token statistics transformer linear time attention via variational rate reduction was accepted spotlight to iclr 2025 september 2024 our paper active dormant attention heads mechanistically demystifying extreme token phenomena in llms was accepted oral to neurips 2024 m3l workshop september 2024 our paper scaling white box transformers for vision was accepted to neurips 2024 may 2024 started a summer research scientist internship at nexusflow may 2024 our new comprehensive paper white box transformers via sparse rate reduction compression is all there is reviewing our white box transformers line of work deriving efficient interpretable and performant transformer like architectures from first principles information theory and signal processing was accepted to jmlr may 2024 our paper a global geometric analysis of maximal coding rate reduction was accepted to icml 2024 january 2024 our paper masked completion via structured diffusion with white box transformers which develops a connection between iterative denoising in diffusion models and representation learning in transformer like deep networks and uses it to construct a performant efficient and interpretable transformer like autoencoder was accepted to iclr 2024 november 2023 our papers emergence of segmentation with minimalistic white box transformers closed loop transcription via convolutional sparse coding and masked completion via structured diffusion with white box transformers were accepted to cpal 2024 october 2023 our paper emergence of segmentation with minimalistic white box transformers was accepted to neurips 2023 xaia workshop september 2023 our paper white box transformers via sparse rate reduction proposing an interpretable and parameter efficient transformer like architecture derived from first principles was accepted to neurips 2023 august 2023 started my ph d program in eecs at uc berkeley 2026 druv pai powered by jekyll academicpages a fork of minimal mistakes