Markov chain importance sampling for minibatches

• Published in: Machine learning 2024-02, Vol.113 (2), p.789-814
• Main Authors: Fuh, Cheng-Der, Wang, Chuan-Ju, Pai, Chen-Hung
• Format: Article
• Language: English
• Subjects: Algorithms; Artificial Intelligence; Artificial neural networks; Computer Science; Control; Importance sampling; Machine Learning; Markov analysis; Markov chains; Mechatronics; Multilayer perceptrons; Natural Language Processing (NLP); Normal distribution; Robotics; Simulation and Modeling
• ISSN: 0885-6125; 1573-0565
• DOI: 10.1007/s10994-023-06489-5

This study investigates importance sampling under the scheme of minibatch stochastic gradient descent, under which the contributions are twofold. First, theoretically, we develop a neat tilting formula, which can be regarded as a general device for asymptotically optimal importance sampling. Second, practically, guided by the formula, we present an effective algorithm for importance sampling which accounts for the effects of minibatches and leverages the Markovian property of the gradients between iterations. Experiments conducted on artificial data confirm that our algorithm consistently delivers superior performance in terms of variance reduction. Furthermore, experiments carried out on real-world data demonstrate that our method, when paired with relatively straightforward models like multilayer perceptron and convolutional neural networks, outperforms in terms of training loss and testing error.