Joint modeling of causal phrases-sentiments-aspects using Hierarchical Pitman Yor Process

• Published in: Information processing & management 2024-07, Vol.61 (4), p.103753, Article 103753
• Main Authors: Yadavilli, V.R.P.S. Sastry, Seshadri, Karthick, S., Nagesh Bhattu
• Format: Article
• Language: English
• Subjects: Gibbs sampling; Hierarchical Pitman Yor Process; Hierarchical topic models; Long Short Term Memory Network; Sentiment analysis; Topic models; Variational Auto Encoder
• ISSN: 0306-4573
• DOI: 10.1016/j.ipm.2024.103753

Traditional sentiment-aware topic models assume that topic or sentiment transition occurs from either a sentence to the next sentence or from a word to the next word. Such models cannot capture a topic or sentiment transition at phrase boundaries. Further, most of the models adopt a sentiment lexicon to initialize sentiment priors and this approach induces coverage problems. To overcome the above-cited limitations, we have proposed a topic model that extracts aspects, sentiments, and causal phrases simultaneously by leveraging Hierarchical Pitman Yor Process (HPYP) that is modified using a sentiment component, a word-tagger to guide the causal phrase generation and a sentiment prior initialized through a sequential model to address coverage problems. We have evaluated our model on six datasets and found that the proposed model outperforms the baselines in terms of perplexity by 14%, topical coherence by 20%, topic diversity by 5%, sentiment classification task’s accuracy by 4% and, precision, recall and F1 score by 2%. Ablation studies assert that sequence model based sentiment prior initialization results in increasing the accuracy of sentiment classification by 2%. •A first-of-its kind model for joint mining of topics, causal phrases, and sentiments.•A first-of-its-kind adaptation of HPYP for phrase generation using a word tagger.•VAE-LSTM induced prior yielded an improvement of 2% over static sentiment prior.•An increase of 14% w.r.t. perplexity and 20% w.r.t. topical coherence were observed.•An increase of 5% w.r.t. topical diversity and 4% w.r.t. accuracy were achieved.