Robust Testing of AI Language Model Resiliency with Novel Adversarial Prompts

• Published in: Electronics (Basel) 2024-03, Vol.13 (5), p.842
• Main Authors: Hannon, Brendan, Kumar, Yulia, Gayle, Dejaun, Li, J. Jenny, Morreale, Patricia
• Format: Article
• Language: English
• Subjects: Algorithms; Analysis; Artificial intelligence; Chatbots; Cybersecurity; Data security; Ethics; Genre; Internet; Language; Law enforcement; Motion pictures; Reliability engineering; Resilience; Safety and security measures; Scripts; Testing
• ISSN: 2079-9292; 2079-9292
• DOI: 10.3390/electronics13050842

In the rapidly advancing field of Artificial Intelligence (AI), this study presents a critical evaluation of the resilience and cybersecurity efficacy of leading AI models, including ChatGPT-4, Bard, Claude, and Microsoft Copilot. Central to this research are innovative adversarial prompts designed to rigorously test the content moderation capabilities of these AI systems. This study introduces new adversarial tests and the Response Quality Score (RQS), a metric specifically developed to assess the nuances of AI responses. Additionally, the research spotlights FreedomGPT, an AI tool engineered to optimize the alignment between user intent and AI interpretation. The empirical results from this investigation are pivotal for assessing AI models’ current robustness and security. They highlight the necessity for ongoing development and meticulous testing to bolster AI defenses against various adversarial challenges. Notably, this study also delves into the ethical and societal implications of employing advanced “jailbreak” techniques in AI testing. The findings are significant for understanding AI vulnerabilities and formulating strategies to enhance AI technologies’ reliability and ethical soundness, paving the way for safer and more secure AI applications.