Search Results - Kunstner, Frederik

  • Showing 1 - 9 results of 9
Refine Results
  1. 1
    Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models

    Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models by Kunstner, Frederik, Yadav, Robin, Milligan, Alan, Schmidt, Mark, Bietti, Alberto

    Published in arXiv.org
    Get full text
    Article
    Save to List
    Saved in:
  2. 2
    Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent

    Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent by Kunstner, Frederik, Kumar, Raunak, Schmidt, Mark

    Published in arXiv.org
    Get full text
    Article
    Save to List
    Saved in:
  3. 3
    Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking

    Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking by Kunstner, Frederik, Portella, Victor S, Schmidt, Mark, Harvey, Nick

    Published in arXiv.org
    Get full text
    Article
    Save to List
    Saved in:
  4. 4
    Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be

    Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be by Kunstner, Frederik, Chen, Jacques, Jonathan Wilder Lavington, Schmidt, Mark

    Published in arXiv.org
    Get full text
    Article
    Save to List
    Saved in:
  5. 5
    Limitations of the Empirical Fisher Approximation for Natural Gradient Descent

    Limitations of the Empirical Fisher Approximation for Natural Gradient Descent by Kunstner, Frederik, Balles, Lukas, Hennig, Philipp

    Published in arXiv.org
    Get full text
    Article
    Save to List
    Saved in:
  6. 6
    Convergence Rates for the MAP of an Exponential Family and Stochastic Mirror Descent -- an Open Problem

    Convergence Rates for the MAP of an Exponential Family and Stochastic Mirror Descent -- an Open Problem by Rémi Le Priol, Kunstner, Frederik, Scieur, Damien, Lacoste-Julien, Simon

    Published in arXiv.org
    Get full text
    Article
    Save to List
    Saved in:
  7. 7
    BackPACK: Packing more into backprop

    BackPACK: Packing more into backprop by Dangel, Felix, Kunstner, Frederik, Hennig, Philipp

    Published in arXiv.org
    Get full text
    Article
    Save to List
    Saved in:
  8. 8
    Adaptive Gradient Methods Converge Faster with Over-Parameterization (but you should do a line-search)

    Adaptive Gradient Methods Converge Faster with Over-Parameterization (but you should do a line-search) by Vaswani, Sharan, Laradji, Issam, Kunstner, Frederik, Si Yi Meng, Schmidt, Mark, Lacoste-Julien, Simon

    Published in arXiv.org
    Get full text
    Article
    Save to List
    Saved in:
  9. 9
    SLANG: Fast Structured Covariance Approximations for Bayesian Deep Learning with Natural Gradient

    SLANG: Fast Structured Covariance Approximations for Bayesian Deep Learning with Natural Gradient by Mishkin, Aaron, Kunstner, Frederik, Nielsen, Didrik, Schmidt, Mark, Khan, Mohammad Emtiyaz

    Published in arXiv.org
    Get full text
    Article
    Save to List
    Saved in:

Search Tools: