How threat actors can use generative artificial intelligence?
December 02, 2024 
Generative Artificial Intelligence (GAI) is quickly revolutionizing numerous industries, together with cybersecurity, permitting the creation of sensible and personalised content material.
The capabilities that make Generative Artificial Intelligence a robust device for progress additionally make it a major threat within the cyber area. The use of GAI by malicious actors is turning into more and more frequent, enabling them to conduct a variety of cyberattacks. From producing deepfakes to enhancing phishing campaigns, GAI is evolving right into a device for large-scale cyber offenses
GAI has captured the eye of researchers and traders for its transformative potential throughout industries. Sadly, its misuse by malicious actors is altering the cyber threat panorama. Among the many most regarding functions of Generative Artificial Intelligence are the creation of deepfakes and disinformation campaigns, that are already proving to be efficient and harmful.
Deepfakes are media content material—corresponding to movies, photographs, or audio—created utilizing GAI to realistically manipulate faces, voices, and even total occasions. The growing sophistication of those applied sciences has made it more durable than ever to tell apart actual content material from faux. This makes deepfakes a potent weapon for attackers engaged in disinformation campaigns, fraud, or privateness violations.
A study by the Massachusetts Institute of Expertise (MIT) offered in 2019 revealed that deepfakes generated by AI might deceive people as much as 60% of the time. Given the developments in AI since then, it’s seemingly that this share has elevated, making deepfakes an much more important threat. Attackers can use them to manufacture occasions, impersonate influential figures, or create situations that manipulate public opinion.
The use of Generative Artificial Intelligence in disinformation campaigns is not hypothetical. In response to a report by the Microsoft Threat Evaluation Middle (MTAC), Chinese language threat actors are utilizing GAI to conduct affect operations focusing on international nations, together with the USA and Taiwan. By producing AI-driven content material, corresponding to provocative memes, movies, and audio, these actors purpose to exacerbate social divisions and affect voter conduct.
For instance, these campaigns leverage faux social media accounts to submit questions and feedback about divisive inner points within the U.S. The info collected by these operations can present insights into voter demographics, probably influencing election outcomes. Microsoft specialists consider that China’s use of AI-generated content material will increase to affect elections in nations like India, South Korea, and the U.S.
GAI can also be a boon for attackers in search of monetary acquire. By automating the creation of phishing emails, malicious actors can scale their campaigns, producing extremely personalised and convincing messages which can be extra more likely to deceive victims.
An instance of this misuse is the creation of fraudulent social media profiles utilizing GAI. In 2022, the Federal Bureau of Investigation (FBI) warned of an uptick in faux profiles designed to use victims financially. GAI permits attackers to generate not solely sensible textual content but additionally photographs, movies, and audio that make these profiles seem real.
Moreover, platforms like FraudGPT and WormGPT, launched in mid-2023, present instruments particularly designed for phishing and business email compromise (BEC) assaults. For a month-to-month price, attackers can entry refined providers that automate the creation of fraudulent emails, growing the effectivity of their scams.
One other space of concern is the use of GAI to develop malicious code. By automating the technology of malware variants, attackers can evade detection mechanisms employed by main anti-malware engines. This makes it simpler for them to hold out large-scale assaults with minimal effort.
One of the alarming elements of GAI is its potential for automating complicated assault processes. This contains creating instruments for offensive functions, corresponding to malware or scripts designed to use vulnerabilities. GAI fashions can refine these instruments to bypass safety defenses, making assaults extra refined and more durable to detect.
Whereas the malicious use of GAI continues to be in its early levels, it’s gaining traction amongst cybercriminals and state-sponsored actors. The growing accessibility of GAI by “as-a-service” fashions will solely speed up its adoption. These providers enable attackers with minimal technical experience to execute superior assaults, democratizing cybercrime.
As an example, in disinformation campaigns, the affect of GAI is already seen. In phishing and monetary fraud, the use of instruments like FraudGPT demonstrates how attackers can scale their operations. The automation of malware improvement is one other worrying development, because it lowers the barrier to entry for cybercrime.
Main safety corporations, in addition to main GAI suppliers like OpenAI, Google, and Microsoft, are actively engaged on options to mitigate these rising threats. Efforts embrace creating sturdy detection mechanisms for deepfakes, enhancing anti-phishing instruments, and creating safeguards to forestall the misuse of GAI platforms.
Nevertheless, the speedy tempo of technological development implies that attackers are at all times a step forward. As GAI turns into extra refined and accessible, the challenges for defenders will develop exponentially.
Generative Artificial Intelligence is a double-edged sword. Whereas it presents immense alternatives for innovation and progress, it additionally presents important dangers when weaponized by malicious actors. The flexibility to create sensible and personalised content material has already reworked the cyber threat panorama, enabling a brand new period of assaults starting from deepfakes to large-scale phishing campaigns.
Because the expertise evolves, so will its misuse. It’s crucial for governments, companies, and people to acknowledge the potential risks of GAI and take proactive measures to handle them. By collaboration and innovation, we can harness the advantages of GAI whereas mitigating its dangers, making certain that this highly effective device serves humanity reasonably than harming it.
Comply with me on Twitter: @securityaffairs and Facebook and Mastodon
(SecurityAffairs – hacking, generative artificial intelligence)