Mutation Testing of Quantum Programs: A Case Study With Qiskit

As quantum computing is still in its infancy, there is an inherent lack of knowledge and technology to test a quantum program properly. In the classical realm, mutation testing has been successfully used to evaluate how well a program's test suite detects seeded faults (i.e., mutants). In this...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on quantum engineering 2022, Vol.3, p.1-17
Main Authors: Fortunato, Daniel, CAMPOS, JOSE, ABREU, RUI
Format: Article
Language:English
Subjects:
Buildings
Case studies
Codes
Computers
Fault detection
Logic gates
Mutation
Performance evaluation
Quantum computing
quantum mutation testing
quantum software engineering
quantum software testing
Qubit
Qubits (quantum computing)
Software testing
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:As quantum computing is still in its infancy, there is an inherent lack of knowledge and technology to test a quantum program properly. In the classical realm, mutation testing has been successfully used to evaluate how well a program's test suite detects seeded faults (i.e., mutants). In this article, building on the definition of syntactically equivalent quantum operations, we propose a novel set of mutation operators to generate mutants based on qubit measurements and quantum gates. To ease the adoption of quantum mutation testing, we further propose QMutPy, an extension of the well-known and fully automated open-source mutation tool MutPy. To evaluate QMutPy's performance, we conducted a case study on 24 real quantum programs written in IBM's Qiskit library. Furthermore, we show how better test suite coverage and improvements to test assertions can increase the test suites' mutation score and quality. QMutPy has proven to be an effective quantum mutation tool, providing insight into the current state of quantum tests and how to improve them.
ISSN:2689-1808
2689-1808
DOI:10.1109/TQE.2022.3195061