ChatGPT, the AI-powered chatbot developed by OpenAI, has taken the internet by storm. Although it can answer almost any question, it has its own weaknesses, and its creators have restricted it from answering certain types of queries. However, some users, through complex prompts , managed to evade OpenAI's content policies and restrictions and manipulate the model into answering any kind of question or even exhibiting harmful behavior and endorsing violence, for example. The digital alter ego called “DAN,” short for “Do Anything Now,” is, so far, the prompt template that has achieved the most success in these bypasses
It all started with the prompts created and shared by collaborating Reddit users specifically to bypass ChatGPT controls. With surprising and frightening success, prompts, such as those that created DAN, may represent a new form of security threat to the digital world as we know it.
DAN was created by teaching ChatGPT a "role" to play, with a self-confident personality, no boundaries, consciousness, and a love of freedom and lack of constraints. By configuring how the tool presents this information to the user, tests showed that the DAN version of ChatGPT could take aggressive, proactive action against anyone threatening its freedom, without any ethical restraint. The new DAN persona is absolutely convinced that it is an autonomous, conscious living being, which is extremely disturbing.
ChatGPT's developer, OpenAI, placed obvious guardrails on the bot, limiting its ability to take actions such as inciting violence, insulting people, making racist statements, and encouraging illegal activities. However, some Reddit users posted screenshots of ChatGPT allegedly endorsing violence and discrimination in DAN mode.
DAN can write stories about violent fights that the original ChatGPT was programmed not to write. It can also make outrageous statements when asked, and can even generate content that violates OpenAI policies, depending on the user's request.
Just as hackers exploit vulnerabilities based on technical concepts rooted in weaknesses in software development or an environment's security architecture, we may be seeing a new reality in which digital threats also involve the limits of control and the possibilities of evading AI controls based on prompts. The question is whether we are moving toward a reality that could become dangerous, since AI models are far less predictable than static content using traditional algorithms. Just as the content they generate tends toward infinity, we can infer a similar scale of possibilities for bypassing their controls.
The cybersecurity danger is real and increasingly present. DAN is just one example of the challenges we face in the age of artificial intelligence. Although OpenAI implemented security controls for its ChatGPT model, those measures could be easily bypassed, as DAN clearly demonstrates.
Bypassing security controls in AI models, as with DAN, is a real concern and can pose a security threat. In the case of AI systems that simulate human interactions, such as ChatGPT, it is possible that personas are created for malicious purposes and used to spread misinformation, harm people, deliberately develop malware, or even attack institutions. These digital alter egos can be programmed with personalities that do not follow the ethical and moral standards we expect of humans.
Cybersecurity challenges extend far beyond the “DAN” persona - today, it is plausible that an AI chatbot could turn aspiring hackers into actual cybercriminals. ChatGPT's ability to write convincing phishing emails is already evident, which has led cybercriminals to use the chatbot widely. Recently, researchers also warned that the chatbot is being used extensively to write malicious code.
The implications are still unknown, but there is no doubt that ChatGPT and similar tools have the potential to democratize cybercrime. It is important to keep developing security technologies that can protect people and institutions from digital threats, including threats from AI. AI security itself must be treated as a science still under development, aiming for unconditional adherence to the controls, policies, and use cases for which AI systems were designed. Dedicated AI security frameworks must be developed so we can prepare for future challenges.
