WithSecure security researchers demonstrated how the GPT-3 natural language generation model and the ChatGPT can be used to carry out phishing scams.<\/p>\n
WithSecure ran a number of experiments to see how modifying the language model’s input influenced the text output. These included phishing and spear-phishing, harassment, social validation for scams, the appropriation of a written style, the creation of intentionally divisive opinions, using the models to create prompts for malicious text, and fake news.<\/p>\n
GPT-3 even created a convincing email thread to use in a phishing campaign, as well as social media posts with hashtags to harass a fictitious CEO of a robotics company. It made use of prompt engineering, a concept related to large language models that entails discovering inputs that produce desirable or useful results in order to generate a variety of content deemed harmful by the researchers.<\/p>\n
It also investigated how changes in inputs to the existing models affected the synthetic text output. The goal was to identify how AI-language generation can be abused through malicious and creative prompt engineering, with the hope that the findings will help guide the development of safer large language models in the future.<\/p>\n