Meta tags:
description=Damek Davis’ Academic Website;

Headings (most frequently used words):

papers, damek, davis, publications, preprints, conference, journal, book, chapters, expository, technical, reports,

Text of the page (most frequently used words):
and (66), damek (62), davis (62), the (45), optimization (31), for (29), dmitriy (27), drusvyatskiy (27), with (18), #journal (15), learning (15), stochastic (14), mathematics (14), appear (14), research (13), convergence (12), methods (12), manuscript (11), functions (11), nonsmooth (11), convex (10), 2025 (10), code (9), analysis (9), gradient (9), jiang (8), subgradient (8), computational (8), siam (8), 2026 (8), under (7), order (7), method (7), problems (7), conference (7), writing (7), about (7), 2015 (6), splitting (6), first (6), wotao (6), theory (6), foundations (6), from (6), liwei (6), math (6), that (6), professor (6), algorithm (5), report (5), 2018 (5), high (5), growth (5), yin (5), mathematical (5), operations (5), set (5), descent (5), charisopoulos (5), mateo (5), one (5), programming (5), thread (5), papers (5), wrote (5), took (5), was (5), ucla (4), technical (4), tao (4), 2023 (4), linearly (4), rate (4), short (4), course (4), 2017 (4), 2014 (4), douglas (4), information (4), ieee (4), sharp (4), weakly (4), 2019 (4), two (4), nonconvex (4), vasileios (4), chen (4), twitter (4), kernel (4), spectral (4), lean (4), proof (4), llms (4), could (4), you (4), more (4), notes (4), assistant (4), students (4), video (3), slides (3), finding (3), model (3), based (3), minimization (3), convergent (3), informs (3), schemes (3), 2020 (3), tame (3), science (3), engineering (3), society (3), best (3), student (3), paper (3), prize (3), book (3), proceedings (3), alexander (3), sadovsky (3), isaacson (3), 2013 (3), systems (3), data (3), 2016 (3), forward (3), this (3), arxiv (3), preprints (3), blog (3), yudong (3), díaz (3), low (3), active (3), strict (3), machine (3), semismooth (3), linear (3), stefano (3), soatto (3), vision (3), sparse (3), guarantees (3), sgd (3), libin (3), zhu (3), maryam (3), fazel (3), matrices (3), when (3), help (3), deep (3), formalization (3), hadamard (3), problem (3), can (3), use (3), here (3), llm (3), semester (3), pytorch (3), read (3), 2012 (3), algebra (3), phd (3), developed (3), associate (3), monotone (2), root (2), norm (2), kellie (2), macphee (2), newton (2), reports (2), several (2), weak (2), convexity (2), expository (2), imaging (2), chapters (2), daqing (2), wan (2), power (2), separation (2), compliant (2), control (2), terminal (2), airspace (2), air (2), traffic (2), april (2), current (2), aleksandr (2), aravkin (2), transactions (2), primal (2), dual (2), valued (2), variational (2), scheme (2), applications (2), faster (2), rachford (2), courtney (2), paquette (2), related (2), complexity (2), near (2), points (2), ima (2), lee (2), converges (2), estimation (2), variance (2), reduction (2), benjamin (2), grimmer (2), proximally (2), diaz (2), composite (2), rank (2), proximal (2), avoid (2), probability (2), are (2), 2022 (2), saddle (2), nearly (2), statistics (2), asymptotic (2), optimality (2), approximation (2), covariance (2), jeongyeol (2), kwon (2), wei (2), qian (2), constantine (2), caramanis (2), 2024 (2), global (2), mixtures (2), component (2), adaptive (2), fourth (2), jonathan (2), balzer (2), computer (2), pattern (2), recognition (2), neural (2), regression (2), neurips (2), product (2), efficiently (2), learn (2), what (2), objective (2), reasoning (2), reinforcement (2), counting (2), partial (2), cubic (2), regime (2), multi (2), competition (2), terry (2), meeting (2), february (2), interview (2), progress (2), received (2), together (2), helped (2), has (2), december (2), think (2), know (2), them (2), public (2), spring (2), why (2), may (2), started (2), appreciate (2), style (2), 2010 (2), loved (2), not (2), have (2), applied (2), college (2), worked (2), music (2), guitar (2), played (2), website (2), teaching (2), email (2), johns (2), hopkins (2), university (2), cornell (2), selected (2), nsf (2), fall (2), github (2), smart, aggregated, cam, log, projecting, onto, ordered, weighted, ell_1, ball, gauss, ill, conditioned, today, siag, opt, news, views, geometry, communication, winner, summary, american, 139, 2011, 823, 834, factorial, noetherian, subrings, series, rings, transportation, part, emerging, technologies, 157, 176, optimal, routing, scheduled, arrivals, dynamic, measurement, 136, 041027, efficient, computation, speed, advisories, arriving, aerospace, vol, 234, 257, tactical, scheduling, precision, past, rajiv, kumar, oscar, lópez, felix, herrmann, beating, level, seismic, interpolation, alternating, approach, three, operator, its, rates, relaxed, peaceman, admm, regularity, assumptions, luis, briceño, arias, backward, half, solving, inclusions, combination, supplementary, note, mirror, young, researchers, stationary, stochastically, 1803, 06523, 1802, 02988, numerical, landscape, phase, retrieval, sham, kakade, jason, trimmed, statistical, via, guided, graphical, subgradients, inference, robust, bilinear, sensing, lijun, ding, matrix, recovery, good, conditioning, rapid, 2021, saddles, lin, xiao, junyu, zhang, confidence, conservative, derivatives, equivalent, semialgebraic, maps, escaping, moreau, envelope, superlinearly, algorithms, geometric, step, decay, converge, quadratic, annals, normality, kaizheng, wang, bernoulli, clustering, mixture, gaussians, unknown, nhat, regressions, xingyuan, minh, pham, elisa, negrini, stanley, osher, jianwei, miao, physical, review, microscopy, beyond, perfect, lenses, zhan, shi, over, smooth, sets, manifolds, stratifications, local, minima, stepsize, asymmetric, approximations, large, scale, visual, search, jingming, dong, nikos, karianakis, joshua, hernandez, multiview, feature, brent, edmunds, madeleine, udell, processing, sound, apalm, clapping, asynchronous, palm, tat, swati, padmanabhan, guanghao, oral, presentation, top, sampling, lipschitz, dimensions, chaoyue, liu, mikhail, belkin, aiming, towards, minimizers, fast, overparametrized, abhishek, roy, krishna, balasubramanian, sen, online, bound, random, fourier, walsh, iteratively, reweighted, machines, recht, updates, sam, dimensional, sub, gaussian, comparison, constant, forbidden, subgraphs, divisor, graphs, erdős, divisibility, average, outer, provably, recovers, central, subspace, index, models, publications, convinced, eventually, accelerate, since, then, organizing, stage, see, whether, boost, performance, through, human, readable, cheat, sheets, initial, tests, ability, implication, graph, equational, theories, project, sair, foundation, while, visiting, ipam, kickoff, view, snippet, gave, talk, paata, ivanisvili, serve, benchmark, inspire, mathematicians, curating, collection, constants, aims, make, tools, far, released, conjecture, grant, sort, completed, aristotle, changed, but, need, check, complete, formalize, almost, spoken, than, anyone, else, coding, even, negotiating, bills, minerva, became, interested, continued, completely, revised, decided, november, october, july, september, scratch, been, thinking, version, improves, become, comfortable, rejection, books, influenced, second, taught, how, structure, text, busy, reader, clear, simple, truth, went, grad, school, pure, without, encouragement, would, faculty, jobs, mentioned, class, solved, open, tried, prove, each, theorem, before, reading, nesterov, year, saw, everywhere, just, understood, nothing, got, excited, joined, lab, learned, thing, theorist, bill, 2006, irvine, 2009, commutative, calculus, realized, intended, study, lived, arts, dorm, only, afford, application, fee, close, beach, throughout, highschool, cleaning, tables, bagging, groceries, making, coffees, bands, recorded, regularly, into, mostly, play, piano, now, like, sing, sight, chord, charts, off, ultimate, banjo, mainly, clawhammer, html, made, pokémon, please, sparingly, correspondence, consulting, questions, other, professional, inquiries, sporadically, post, ben, seattle, vasilis, purdue, meta, postdoc, graduated, positions, advised, penn, wish, discuss, advising, collaboration, send, concise, informative, advisor, who, work, tend, energy, levels, match, exceed, mine, hierarchical, concept, point, showed, characterized, distribution, proved, which, covers, essentially, all, networks, exponential, accelerations, identified, collaborators, works, activity, group, career, award, sloan, fellowship, awards, teach, practice, sometimes, write, lecture, interests, wharton, department, previously, orie, postdoctoral, fellow, alibaba, aws, long, term, visitor, simon, institute, bridging, discrete, continuous, program, currently, editor, scholar, old,

Text of the page (random words):
4 llm program i am currently an associate editor at mathematical programming and foundations of computational mathematics research interests optimization and machine learning ai for math teaching i teach theory and practice of optimization and machine learning i sometimes write lecture notes e g optimization in pytorch and convex analysis and first order methods selected awards i received a sloan research fellowship in mathematics an nsf career award and the siam activity group on optimization best paper prize selected works read more about my research here together with collaborators i identified when why spectral gradient methods help in deep learning developed exponential accelerations of gradient descent semismooth newton and the subgradient method proved first guarantees for sgd on weakly convex and tame functions which covers essentially all neural networks characterized the asymptotic distribution of sgd in nonsmooth optimization developed the concept of a strict saddle point in nonsmooth optimization and showed proximal methods and stochastic subgradient methods avoid them developed kernel methods that efficiently learn sparse hierarchical functions students i ve advised 5 phd students if you are a penn student and wish to discuss advising collaboration send me a concise informative email to set up a meeting i am an active advisor students who work best with me tend to have energy levels that match or exceed mine graduated phd students from cornell current positions tao jiang meta postdoc liwei jiang purdue assistant professor vasilis charisopoulos uw seattle assistant professor mateo díaz johns hopkins university assistant professor ben grimmer johns hopkins university assistant professor blog i sporadically post notes here please use my email sparingly for correspondence related to consulting research questions teaching or other professional inquiries you may not know that i started programming in html in 98 i made a pokémon website i took up guitar in 99 played in bands wrote and recorded music regularly into college i played banjo from 2010 2012 mainly in clawhammer style i mostly play piano now i like to sing and sight read chord charts off of ultimate guitar i worked throughout highschool cleaning tables bagging groceries and making coffees in 2006 i applied to one college uc irvine i could only afford one application fee and it was close to the beach i intended to study music and lived in the arts dorm i took calculus my first semester and realized i loved math i took algebra with daqing wan in 2009 we worked on a commutative algebra problem i went to grad school at ucla for pure math in 2010 i loved algebra i learned ml was a thing in 2012 and took learning from data with set theorist bill chen i got excited about ai and joined ucla s vision lab in 2012 a year in i saw optimization was everywhere i just understood nothing about it i read nesterov s book in 2013 i tried to prove each theorem before reading the proof i took wotao s course in f 2013 and solved an open problem he mentioned in class without wotao s encouragement i would not have applied to faculty jobs in 2014 i became interested in writing in 2016 two books influenced me clear and simple as the truth and style the first helped me appreciate writing the second taught me how to structure text so a busy reader could appreciate it i think writing improves when you become more comfortable with rejection i decided to do more writing in public in 2025 in the spring semester i wrote the 2025 version of my course notes for optimization in pytorch in may i started writing notes on what i ve been thinking about in july september i wrote about learning llm engineering from scratch in october i wrote about the objective of reasoning with reinforcement learning in november and december i wrote about why spectral gradient methods help in deep learning i continued writing in public in 2026 in spring semester i completely revised my course on optimization in pytorch minerva convinced me llms could eventually accelerate mathematics research since then i ve spoken to llms more than anyone else i know i ve use them for math coding writing and even negotiating bills llms sort of helped me almost formalize convergence of gradient descent in lean as of april 2025 no llm could complete the proof as of december 2025 i think this has changed but need to check as of february 2026 aristotle has completed the proof i received a grant that aims to make progress on the hadamard conjecture with rl tools so far i ve released a paper on counting partial hadamard matrices in the cubic regime thread together with a lean formalization with paata ivanisvili and terry tao i m curating a collection of optimization constants in mathematics that can serve as an ai benchmark inspire progress by mathematicians i gave a talk on ai for math and an interview while visiting the ipam ai for science kickoff meeting in february 2026 you can view a short snippet from that interview here with terry tao and the sair foundation i m co organizing a multi stage competition to see whether one can boost the performance of weak llms on mathematics problems through the use of short human readable cheat sheets the initial competition tests this ability on the implication graph of the equational theories project publications preprints conference papers journal papers book chapters expository reports preprints average gradient outer product in kernel regression provably recovers the central subspace for multi index models libin zhu damek davis dmitriy drusvyatskiy maryam fazel manuscript 2026 forbidden subgraphs in divisor graphs and an erdős divisibility problem damek davis manuscript 2026 a short proof of near linear convergence of adaptive gradient descent under fourth order growth and convexity damek davis and dmitriy drusvyatskiy manuscript 2026 the sharp one dimensional convex sub gaussian comparison constant damek davis and sam power manuscript 2026 twitter thread lean formalization counting partial hadamard matrices in the cubic regime damek davis manuscript 2026 twitter thread lean formalization when do spectral gradient updates help in deep learning damek davis and dmitriy drusvyatskiy manuscript 2025 what is the objective of reasoning with reinforcement learning damek davis benjamin recht manuscript 2025 iteratively reweighted kernel machines efficiently learn sparse functions libin zhu damek davis dmitriy drusvyatskiy maryam fazel manuscript 2025 spectral norm bound for the product of random fourier walsh matrices libin zhu damek davis dmitriy drusvyatskiy maryam fazel manuscript 2025 conference papers online covariance estimation in nonsmooth stochastic approximation liwei jiang abhishek roy krishna balasubramanian damek davis dmitriy drusvyatskiy sen na in conference on learning theory 2025 aiming towards the minimizers fast convergence of sgd for overparametrized problems chaoyue liu dmitriy drusvyatskiy mikhail belkin damek davis yi an ma neurips 2023 a gradient sampling method with complexity guarantees for lipschitz functions in high and low dimensions damek davis dmitriy drusvyatskiy yin tat lee swati padmanabhan guanghao ye neurips 2022 oral presentation top 1 high probability guarantees for stochastic convex optimization damek davis dmitriy drusvyatskiy in conference on learning theory 2020 global convergence of em algorithm for mixtures of two component linear regression jeongyeol kwon wei qian constantine caramanis yudong chen and damek davis conference on learning theory 2019 the sound of apalm clapping faster nonsmooth nonconvex optimization with stochastic asynchronous palm damek davis brent edmunds madeleine udell neural information processing systems 2016 report multiview feature engineering and learning jingming dong nikos karianakis damek davis joshua hernandez jonathan balzer and stefano soatto in proceedings of the ieee conference on computer vision and pattern recognition 2015 asymmetric sparse kernel approximations for large scale visual search damek davis jonathan balzer stefano soatto in proceedings of the ieee conference on computer vision and pattern recognition 2014 journal papers gradient descent with adaptive stepsize converges nearly linearly under fourth order growth damek davis dmitriy drusvyatskiy liwei jiang mathematical programming 2025 active manifolds stratifications and convergence to local minima in nonsmooth optimization damek davis dmitriy drusvyatskiy liwei jiang foundations of computational mathematics to appear stochastic optimization over proximally smooth sets damek davis dmitriy drusvyatskiy zhan shi siam journal on optimization to appear computational microscopy beyond perfect lenses xingyuan lu minh pham elisa negrini damek davis stanley j osher jianwei miao physical review e to appear global optimality of the em algorithm for mixtures of two component linear regressions jeongyeol kwon wei qian constantine caramanis yudong chen damek davis nhat ho ieee transactions on information theory 2024 clustering a mixture of gaussians with unknown covariance damek davis mateo diaz kaizheng wang bernoulli to appear asymptotic normality and optimality in nonsmooth stochastic approximation damek davis dmitriy drusvyatskiy liwei jiang the annals of statistics to appear a nearly linearly convergent first order method for nonsmooth functions with quadratic growth damek davis liwei jiang foundations of computational mathematics to appear code twitter thread stochastic algorithms with geometric step decay converge linearly on sharp functions damek davis dmitriy drusvyatskiy vasileios charisopoulos mathematical programming to appear code a superlinearly convergent subgradient method for sharp semismooth problems vasileios charisopoulos damek davis mathematics of operations research 2023 code twitter thread escaping strict saddle points of the moreau envelope in nonsmooth optimization damek davis mateo díaz dmitriy drusvyatskiy siam journal on optimization 2022 variance reduction for root finding problems damek davis mathematical programming to appear conservative and semismooth derivatives are equivalent for semialgebraic maps damek davis dmitriy drusvyatskiy set valued and variational analysis to appear from low probability to high confidence in stochastic convex optimization damek davis dmitriy drusvyatskiy lin xiao junyu zhang journal of machine learning research to appear proximal methods avoid active strict saddles of weakly convex functions damek davis dmitriy drusvyatskiy foundations of computational mathematics 2021 low rank matrix recovery with composite optimization good conditioning and rapid convergence vasileios charisopoulos yudong chen damek davis mateo díaz lijun ding dmitriy drusvyatskiy foundations of computational mathematics to appear code composite optimization for robust rank one bilinear sensing vasileios charisopoulos damek davis mateo diaz dmitriy drusvyatskiy ima journal on information and inference 2020 code graphical convergence of subgradients in nonconvex optimization and learning damek davis dmitriy drusvyatskiy mathematics of operations research to appear proximally guided stochastic subgradient method for nonsmooth nonconvex problems damek davis benjamin grimmer siam journal on optimization to appear code trimmed statistical estimation via variance reduction aleksandr aravkin damek davis mathematics of operations research 2019 video stochastic subgradient method converges on tame functions damek davis dmitriy drusvyatskiy sham kakade jason d lee foundations of computational mathematics to appear the nonsmooth landscape of phase retrieval damek davis dmitriy drusvyatskiy courtney paquette ima journal on numerical analysis 2018 stochastic model based minimization of weakly convex functions damek davis dmitriy drusvyatskiy siam journal on optimization 2019 blog this is the combination of the two arxiv preprints arxiv 1802 02988 and arxiv 1803 06523 supplementary technical note complexity of finding near stationary points of convex functions stochastically related report on nonsmooth nonconvex mirror descent stochastic model based minimization under high order growth 2018 informs optimization society young researchers prize 2019 subgradient methods for sharp weakly convex functions damek davis dmitriy drusvyatskiy kellie j macphee courtney paquette journal of optimization theory and applications 2018 forward backward half forward algorithm for solving monotone inclusions luis m briceño arias damek davis siam journal on optimization 2018 convergence rate analysis of the forward douglas rachford splitting scheme damek davis siam journal on optimization 2015 convergence rate analysis of primal dual splitting schemes damek davis siam journal on optimization 2015 faster convergence rates of relaxed peaceman rachford and admm under regularity assumptions damek davis wotao yin mathematics of operations research 2016 a three operator splitting scheme and its optimization applications damek davis wotao yin set valued and variational analysis 2017 code slides beating level set methods for 5d seismic data interpolation a primal dual alternating approach rajiv kumar oscar lópez damek davis aleksandr y aravkin felix j herrmann ieee transactions on computational imaging 2017 tactical scheduling for precision air traffic operations past research and current problems douglas r isaacson alexander v sadovsky damek davis journal of aerospace information systems april vol 11 no 4 pp 234 257 efficient computation of separation compliant speed advisories for air traffic arriving in terminal airspace alexander v sadovsky damek davis douglas r isaacson journal of dynamic systems measurement and control 136 4 041027 2014 separation compliant optimal routing and control of scheduled arrivals in a terminal airspace alexander v sadovsky damek davis and douglas r isaacson transportation research part c emerging technologies 37 2013 157 176 factorial and noetherian subrings of power series rings damek davis daqing wan proceedings of the american mathematical society 139 2011 no 3 823 834 book chapters convergence rate analysis of several splitting schemes damek davis wotao yin splitting methods in communication and imaging science and engineering 2017 video slides summary winner of the 2014 informs optimization society best student paper prize expository a short course on convex analysis and first order methods damek davis manuscript 2023 subgradient methods under weak convexity and tame geometry damek davis dmitriy drusvyatskiy siag opt news and views 2020 convergence rate analysis of several splitting schemes damek davis informs os today 2015 technical reports a linearly convergent gauss newton subgradient method for ill conditioned problems damek davis tao jiang technical report 2023 code stochastic model based minimization under high order growth damek davis dmitriy drusvyatskiy kellie j macphee technical report 2018 an o n log n algorithm for project...