Research Interests

convex/non-convex optimization, stochastic methods, adaptive algorithms, online learning, machine learning

Education

• B.S. in Computer Engineering (Salutatorian), Bilkent University. 2012-2017
• Ph.D in Computer and Communication Sciences, EPFL. 09/2017 - 08/2023
• Postdoc at IFML, UT Austin. 11/2023 - Present

Employment

• Postdoctoral Fellow at University of Texas at Austin. 11/2023 - Preent
  • Department of Electrical and Computer Engineering
  • Supervisors: Sujay Sanghavi and Aryan Mokhtari
• Research Assistant at EPFL, Switzerland. 09/2017 - 08/2023
  • School of Computer and Communication Sciences
  • Supervisor: Volkan Cevher

Honors and Awards

• Swiss National Science Foundation Postdoc.Mobility Return Grant (CHF 115K), 2026-2027
• Swiss National Science Foundation Postdoc.Mobility Grant (CHF 120K), 2023-2025
• EPFL IC Doctoral Studies Fellowship, 2017-2018
• Spotlight Paper at NeurIPS 2018, 2019, 2023 and ICML 2025
• Top Reviewer at NeurIPS 2024 and ICML 2025
• Travel Award at Neurips, 2019
• Comprehensive Scholarship (Tuition waiver with monthly stipend), Bilkent University, 2012-2017

Professional Service

• Reviewer Duty : NeurIPS, ICML, COLT, ICLR, OJMO, JMLR, Springer Machine Learning, IEEE Transactions on Information Theory

• Tutorial at EUSIPCO 2021. 01/2021
  • Adaptive Optimization Methods for Machine Learning and Signal Processing
  • Lecturers: Ali Kavis, Ahmet Alacaoglu, Kfir Levy, Volkan Cevher
• Minisymposium at SIAM OP23. 05/2023
  • Adaptivity and Universality: First-order Methods and Beyond
  • Organizers: Volkan Cevher, Ali Kavis, Kimon Antonakopoulos

Invited Talks

• INFORMS 2025 (at Atlanta). Parameter-free second-order methods for min-max optimization.
• ICCOPT 2025 (at USC). Parameter-free second-order methods for min-max optimization.
• EUROPT 2025 (at University of Southampton). Parameter-free second-order methods for min-max optimization.
• IFML Symposium Workshop (at Simons Institute) 2024. Parameter-free second-order methods for min-max optimization.
• ODI Group - Niao He (at ETH Zurich) 2024. Parameter-free second-order methods for min-max optimization.
• IFML Seminar (at UT Austin) 2024. Designing fast, parameter-free algorithms for convex optimization.
• SIAM Conference on Optimization 2023. A first approach to noise-adaptive accelerated second-order methods.
• SIAM MDS Conference 2022. High probability convergence of AdaGrad for non-convex optimization.

Publications

Online Learning-guided Learning Rate Adaptation via Gradient Alignment

R. Jiang*, A. Kavis*, , A. Mokhtari*. Preprint.

Upweighting Easy Samples in Fine-Tuning Mitigates Forgetting

S. Sanyal*, H. Prairie*, R. Das*, A. Kavis*, S. Sanghavi. ICML 2025 (Spotlight).

Understanding Self-Supervised Learning via Gaussian Mixture Models

P. Bansal, A. Kavis, S. Sanghavi. NeurIPS 2025.

Adaptive and Optimal Second-order Optimistic Methods for Minimax Optimization

R. Jiang, A. Kavis, Q. Jin, S. Sanghavi, A. Mokhtari. NeurIPS 2024.

Advancing the lower bounds: An accelerated, stochastic, second-order method with optimal adaptation to inexactness

A. Agafonov, D. Kamzolov, A. Gasnikov, A. Kavis, K. Antonakopoulos, V. Cevher, M. Takáč. ICLR 2024.

Universal Gradient Methods for Stochastic Convex Optimization

A. Rodomanov, A. Kavis, Y. Wu, K. Antonakopoulos, V. Cevher. ICML 2024.

Alternation makes the adversary weaker in two-player games

V. Cevher, A. Cutkosky, A. Kavis, G. Piliouras, S. Skoulakis, L. Viano. NeurIPS 2023.

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

A. Kavis*, S. Skoulakis*, K. Antonakopoulos, L. T. Dadi, V. Cevher. NeurIPS 2022.

Extra-Newton: A First Approach to Noise-Adaptive Accelerated Second-Order Methods

K. Antonakopoulos*, A. Kavis*, V. Cevher. NeurIPS 2022.

High Probability Bounds for a Class of Non-convex Algorithms with AdaGrad Stepsize

A. Kavis, K. Y. Levy, V. Cevher. ICLR 2022.

STORM+: Fully Adaptive SGD with Recursive Momentum for Nonconvex Optimization

K. Y. Levy, A. Kavis, V. Cevher. NeurIPS 2021.

Sifting through the noise: Universal first-order methods for stochastic variational inequalities

K. Antonakopoulos, T. Pethick, A. Kavis, P. Mertikopoulos, V. Cevher. NeurIPS 2021.

On the almost sure convergence of stochastic gradient descent in non-convex problems

P. Mertikopoulos, N. Hallak, A. Kavis, V. Cevher. NeurIPS 2020.

Double-Loop Unadjusted Langevin Algorithm

P. T. Y. Rolland, A. Eftekhari, A. Kavis, V. Cevher. ICML 2020.

UniXGrad: A Universal, Adaptive Algorithm with Optimal Guarantees for Constrained Optimization

A. Kavis∗, K. Y. Levy∗, F. Bach, V. Cevher. NeurIPS 2019.

Efficient learning of smooth probability functions from Bernoulli tests with guarantees

P. T. Y. Rolland, A. Kavis, A. Immer, A. Singla, V. Cevher. ICML 2019.

Mirrored Langevin Dynamics

Y. Hsieh, A. Kavis, P. Rolland, V. Cevher. NeurIPS 2018.