Intelligent failure prediction models for scientific workflows

• Published in: Expert systems with applications 2015-02, Vol.42 (3), p.980-989
• Main Authors: Bala, Anju, Chana, Inderveer
• Format: Article
• Language: English
• Subjects: Applied sciences; Cloud Computing; Computation; Computer science; control theory; systems; Computer systems and distributed systems. User interface; Computer systems performance. Reliability; Exact sciences and technology; Failure; Failure analysis; Failure prediction; Information systems. Data bases; Machine learning; Mathematical models; Memory organisation. Data processing; Programming languages; Scientific workflows; Software; Tasks; Workflow; Workflows
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2014.09.014

•Intelligent task failure models using machine learning approaches are proposed.•The accuracy of proposed models is validated in Pegasus and Amazon EC2.•The prediction accuracy of (94%) is achieved using Naïve Bayes approach. The ever-growing demand and heterogeneity of Cloud Computing is garnering popularity with scientific communities to utilize the services of Cloud for executing large scale scientific applications in the form of set of tasks known as Workflows. As scientific workflows stipulate a process or computation to be executed in the form of data flow and task dependencies that allow users to simply articulate multi-step computational and complex tasks. Hence, proactive fault tolerance is required for the execution of scientific workflows. To reduce the failure effect of workflow tasks on the Cloud resources during execution, task failures can be intelligently predicted by proactively analyzing the data of multiple scientific workflows using the state of the art of machine learning approaches for failure prediction. Therefore, this paper makes an effort to focus on the research problem of designing an intelligent task failure prediction models for facilitating proactive fault tolerance by predicting task failures for Scientific Workflow applications. Firstly, failure prediction models have been implemented through machine learning approaches using evaluated performance metrics and also demonstrates the maximum prediction accuracy for Naive Bayes. Then, the proposed failure models have also been validated using Pegasus and Amazon EC2 by comparing actual task failures with predicted task failures.