Pashto poetry generation: deep learning with pre-trained transformers for low-resource languages

• Published in: PeerJ. Computer science 2024-08, Vol.10, p.e2163, Article e2163
• Main Authors: Ullah, Imran, Ullah, Khalil, Khan, Hamad, Aurangzeb, Khursheed, Anwar, Muhammad Shahid, Syed, Ikram
• Format: Article
• Language: English
• Subjects: Analysis; Artificial Intelligence; Bloomz-560m; Computational linguistics; Data Mining and Machine Learning; Deep learning; Electric transformers; LaMini-Cerebras-590M; Language processing; Machine learning; Natural Language and Speech; Natural language interfaces; Natural language processing; Poetry; Poetry generation; Text Mining
• ISSN: 2376-5992; 2376-5992
• DOI: 10.7717/peerj-cs.2163

Generating poetry using machine and deep learning techniques has been a challenging and exciting topic of research in recent years. It has significance in natural language processing and computational linguistics. This study introduces an innovative approach to generate high-quality Pashto poetry by leveraging two pre-trained transformer models, LaMini-Cerebras-590M and bloomz-560m. The models were trained on an extensive new and quality Pashto poetry dataset to learn the underlying complex patterns and structures. The trained models are then used to generate new Pashto poetry by providing them with a seed text or prompt. To evaluate the quality of the generated poetry, we conducted both subjective and objective evaluations, including human evaluation. The experimental results demonstrate that the proposed approach can generate Pashto poetry that is comparable in quality to human-generated poetry. The study provides a valuable contribution to the field of Pashto language and poetry generation and has potential applications in natural language processing and computational linguistics.