Evaluating the Effectiveness of GPT-4 Turbo in Creating Defeaters for Assurance Cases

Assurance cases (ACs) are structured arguments that support the verification of the correct implementation of systems' non-functional requirements, such as safety and security, thereby preventing system failures which could lead to catastrophic outcomes, including loss of lives. ACs facilitate...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2024-01
Main Authors: Kimya Khakzad Shahandashti, Sivakumar, Mithila, Mohammad Mahdi Mohajer, Belle, Alvine B, Wang, Song, Lethbridge, Timothy C
Format: Article
Language:English
Subjects:
Assurance
Large language models
System failures
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by
cites
container_end_page
container_issue
container_start_page
container_title arXiv.org
container_volume
creator Kimya Khakzad Shahandashti
Sivakumar, Mithila
Mohammad Mahdi Mohajer
Belle, Alvine B
Wang, Song
Lethbridge, Timothy C
description Assurance cases (ACs) are structured arguments that support the verification of the correct implementation of systems' non-functional requirements, such as safety and security, thereby preventing system failures which could lead to catastrophic outcomes, including loss of lives. ACs facilitate the certification of systems in accordance with industrial standards, for example, DO-178C and ISO 26262. Identifying defeaters arguments that refute these ACs is essential for improving the robustness and confidence in ACs. To automate this task, we introduce a novel method that leverages the capabilities of GPT-4 Turbo, an advanced Large Language Model (LLM) developed by OpenAI, to identify defeaters within ACs formalized using the Eliminative Argumentation (EA) notation. Our initial evaluation gauges the model's proficiency in understanding and generating arguments within this framework. The findings indicate that GPT-4 Turbo excels in EA notation and is capable of generating various types of defeaters.
format article
fullrecord <record><control><sourceid>proquest</sourceid><recordid>TN_cdi_proquest_journals_2920919962</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2920919962</sourcerecordid><originalsourceid>FETCH-proquest_journals_29209199623</originalsourceid><addsrcrecordid>eNqNiksKwjAUAIMgWLR3eOC6kKYfzVJq1aWLui6xvGhLSTQv6fkt6AFczcDMgkUiy9JknwuxYjHRwDkX5U4URRaxWz2pMSjfmwf4J0KtNXa-n9AgEVgN52uT5NAEd7fQG6gcfucj6tnQEWjr4EAUnDIdQqUIacOWWo2E8Y9rtj3VTXVJXs6-A5JvBxucmVMrpOAylbIU2X_XBzuFP9Y</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2920919962</pqid></control><display><type>article</type><title>Evaluating the Effectiveness of GPT-4 Turbo in Creating Defeaters for Assurance Cases</title><source>Publicly Available Content Database</source><creator>Kimya Khakzad Shahandashti ; Sivakumar, Mithila ; Mohammad Mahdi Mohajer ; Belle, Alvine B ; Wang, Song ; Lethbridge, Timothy C</creator><creatorcontrib>Kimya Khakzad Shahandashti ; Sivakumar, Mithila ; Mohammad Mahdi Mohajer ; Belle, Alvine B ; Wang, Song ; Lethbridge, Timothy C</creatorcontrib><description>Assurance cases (ACs) are structured arguments that support the verification of the correct implementation of systems' non-functional requirements, such as safety and security, thereby preventing system failures which could lead to catastrophic outcomes, including loss of lives. ACs facilitate the certification of systems in accordance with industrial standards, for example, DO-178C and ISO 26262. Identifying defeaters arguments that refute these ACs is essential for improving the robustness and confidence in ACs. To automate this task, we introduce a novel method that leverages the capabilities of GPT-4 Turbo, an advanced Large Language Model (LLM) developed by OpenAI, to identify defeaters within ACs formalized using the Eliminative Argumentation (EA) notation. Our initial evaluation gauges the model's proficiency in understanding and generating arguments within this framework. The findings indicate that GPT-4 Turbo excels in EA notation and is capable of generating various types of defeaters.</description><identifier>EISSN: 2331-8422</identifier><language>eng</language><publisher>Ithaca: Cornell University Library, arXiv.org</publisher><subject>Assurance ; Large language models ; System failures</subject><ispartof>arXiv.org, 2024-01</ispartof><rights>2024. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://www.proquest.com/docview/2920919962?pq-origsite=primo$$EHTML$$P50$$Gproquest$$Hfree_for_read</linktohtml><link.rule.ids>780,784,25753,37012,44590</link.rule.ids></links><search><creatorcontrib>Kimya Khakzad Shahandashti</creatorcontrib><creatorcontrib>Sivakumar, Mithila</creatorcontrib><creatorcontrib>Mohammad Mahdi Mohajer</creatorcontrib><creatorcontrib>Belle, Alvine B</creatorcontrib><creatorcontrib>Wang, Song</creatorcontrib><creatorcontrib>Lethbridge, Timothy C</creatorcontrib><title>Evaluating the Effectiveness of GPT-4 Turbo in Creating Defeaters for Assurance Cases</title><title>arXiv.org</title><description>Assurance cases (ACs) are structured arguments that support the verification of the correct implementation of systems' non-functional requirements, such as safety and security, thereby preventing system failures which could lead to catastrophic outcomes, including loss of lives. ACs facilitate the certification of systems in accordance with industrial standards, for example, DO-178C and ISO 26262. Identifying defeaters arguments that refute these ACs is essential for improving the robustness and confidence in ACs. To automate this task, we introduce a novel method that leverages the capabilities of GPT-4 Turbo, an advanced Large Language Model (LLM) developed by OpenAI, to identify defeaters within ACs formalized using the Eliminative Argumentation (EA) notation. Our initial evaluation gauges the model's proficiency in understanding and generating arguments within this framework. The findings indicate that GPT-4 Turbo excels in EA notation and is capable of generating various types of defeaters.</description><subject>Assurance</subject><subject>Large language models</subject><subject>System failures</subject><issn>2331-8422</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>PIMPY</sourceid><recordid>eNqNiksKwjAUAIMgWLR3eOC6kKYfzVJq1aWLui6xvGhLSTQv6fkt6AFczcDMgkUiy9JknwuxYjHRwDkX5U4URRaxWz2pMSjfmwf4J0KtNXa-n9AgEVgN52uT5NAEd7fQG6gcfucj6tnQEWjr4EAUnDIdQqUIacOWWo2E8Y9rtj3VTXVJXs6-A5JvBxucmVMrpOAylbIU2X_XBzuFP9Y</recordid><startdate>20240131</startdate><enddate>20240131</enddate><creator>Kimya Khakzad Shahandashti</creator><creator>Sivakumar, Mithila</creator><creator>Mohammad Mahdi Mohajer</creator><creator>Belle, Alvine B</creator><creator>Wang, Song</creator><creator>Lethbridge, Timothy C</creator><general>Cornell University Library, arXiv.org</general><scope>8FE</scope><scope>8FG</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>HCIFZ</scope><scope>L6V</scope><scope>M7S</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope></search><sort><creationdate>20240131</creationdate><title>Evaluating the Effectiveness of GPT-4 Turbo in Creating Defeaters for Assurance Cases</title><author>Kimya Khakzad Shahandashti ; Sivakumar, Mithila ; Mohammad Mahdi Mohajer ; Belle, Alvine B ; Wang, Song ; Lethbridge, Timothy C</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-proquest_journals_29209199623</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Assurance</topic><topic>Large language models</topic><topic>System failures</topic><toplevel>online_resources</toplevel><creatorcontrib>Kimya Khakzad Shahandashti</creatorcontrib><creatorcontrib>Sivakumar, Mithila</creatorcontrib><creatorcontrib>Mohammad Mahdi Mohajer</creatorcontrib><creatorcontrib>Belle, Alvine B</creatorcontrib><creatorcontrib>Wang, Song</creatorcontrib><creatorcontrib>Lethbridge, Timothy C</creatorcontrib><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>Materials Science &amp; Engineering Collection</collection><collection>ProQuest Central (Alumni)</collection><collection>ProQuest Central</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Engineering Collection</collection><collection>Engineering Database</collection><collection>Publicly Available Content Database</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering Collection</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Kimya Khakzad Shahandashti</au><au>Sivakumar, Mithila</au><au>Mohammad Mahdi Mohajer</au><au>Belle, Alvine B</au><au>Wang, Song</au><au>Lethbridge, Timothy C</au><format>book</format><genre>document</genre><ristype>GEN</ristype><atitle>Evaluating the Effectiveness of GPT-4 Turbo in Creating Defeaters for Assurance Cases</atitle><jtitle>arXiv.org</jtitle><date>2024-01-31</date><risdate>2024</risdate><eissn>2331-8422</eissn><abstract>Assurance cases (ACs) are structured arguments that support the verification of the correct implementation of systems' non-functional requirements, such as safety and security, thereby preventing system failures which could lead to catastrophic outcomes, including loss of lives. ACs facilitate the certification of systems in accordance with industrial standards, for example, DO-178C and ISO 26262. Identifying defeaters arguments that refute these ACs is essential for improving the robustness and confidence in ACs. To automate this task, we introduce a novel method that leverages the capabilities of GPT-4 Turbo, an advanced Large Language Model (LLM) developed by OpenAI, to identify defeaters within ACs formalized using the Eliminative Argumentation (EA) notation. Our initial evaluation gauges the model's proficiency in understanding and generating arguments within this framework. The findings indicate that GPT-4 Turbo excels in EA notation and is capable of generating various types of defeaters.</abstract><cop>Ithaca</cop><pub>Cornell University Library, arXiv.org</pub><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier EISSN: 2331-8422
ispartof arXiv.org, 2024-01
issn 2331-8422
language eng
recordid cdi_proquest_journals_2920919962
source Publicly Available Content Database
subjects Assurance
Large language models
System failures
title Evaluating the Effectiveness of GPT-4 Turbo in Creating Defeaters for Assurance Cases
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-06T21%3A00%3A33IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest&rft_val_fmt=info:ofi/fmt:kev:mtx:book&rft.genre=document&rft.atitle=Evaluating%20the%20Effectiveness%20of%20GPT-4%20Turbo%20in%20Creating%20Defeaters%20for%20Assurance%20Cases&rft.jtitle=arXiv.org&rft.au=Kimya%20Khakzad%20Shahandashti&rft.date=2024-01-31&rft.eissn=2331-8422&rft_id=info:doi/&rft_dat=%3Cproquest%3E2920919962%3C/proquest%3E%3Cgrp_id%3Ecdi_FETCH-proquest_journals_29209199623%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2920919962&rft_id=info:pmid/&rfr_iscdi=true