A weighted metric scalarization approach for multiobjective BOHB hyperparameter optimization in LSTM model for sentiment analysis

In recent years, hyperparameter tuning has become increasingly essential, especially for neural network models, where models with good performance generally require a long time to train. In contrast, fast-trained models often result in poor model performance. Thus, it is essential to determine the o...

Full description

Saved in:
Bibliographic Details
Published in:Information sciences 2023-10, Vol.644, p.119282, Article 119282
Main Authors: Andhika Viadinugroho, Raden Aurelius, Rosadi, Dedi
Format: Article
Language:English
Subjects:
BOHB
LSTM
Multiobjective optimization
Weighted metric
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In recent years, hyperparameter tuning has become increasingly essential, especially for neural network models, where models with good performance generally require a long time to train. In contrast, fast-trained models often result in poor model performance. Thus, it is essential to determine the optimal hyperparameter configuration for a neural network model that can produce good model performance and fast training time. This paper presents a multiobjective optimization for Long-Short Term Memory (LSTM) model using Bayesian optimization-Hyperband (BOHB) with a weighted metric scalarization method. The objective functions used are the F1-score (Macro) and model training time, where both of the objective functions are considered equally important. The data used in this paper are the SmSA and EmoT datasets taken from the IndoNLU benchmark. The results are optimized model using multiobjective BOHB optimization with weighted metric scalarization method produced higher F1-score and faster model training time in both datasets compared to the baseline model and optimized model using single objective BOHB. •This paper implements the weighted metric scalarization for the multiobjective BOHB optimization method.•We tested our proposed method to optimize the LSTM model hyperparameter for sentiment analysis.•The data used in this paper are the SmSA and EmoT datasets taken from the IndoNLU benchmark.•We compare our proposed method against the original Bayesian optimization, Hyperband, and BOHB.•The optimized LSTM model using our proposed method produced higher F1-score and faster training time, simultaneously.
ISSN:0020-0255
DOI:10.1016/j.ins.2023.119282