Investigating AI in defending against deepfake-based phishing attacks 

By Avaneesh Solapurkar 

1. Abstract 

The rise of AI has led to the revolutionization of both daily life and specialized fields like cybersecurity. However, AI can be leveraged for cyberattacks by various methods, such as writing AI-generated code for malware, writing convincing phishing messages, and using deepfakes to give phishing attacks a greater effect. Deepfakes can be used in phishing attacks because humans tend to trust faces and audio, leading to a greater chance for attackers to receive money from victims who likely thought they were sending money to a friend, family member, or coworker. This research paper aims to shed light on deepfake-based phishing attacks because they could become more prominent in the future with further developments in AI, using existing sources as cited in the bibliography. It concludes with claiming that AI should be used to defend against deepfake-based phishing attacks. 

2. Introduction 

Artificial Intelligence (AI) models are an increasing threat to current cyber-defense systems, with McKinsey & Company stating that "[AI cyberattacks'] breakout times [are] now under an hour" (Lewis et al., 2025). Specifically, deepfakes and phishing attacks have been supercharged thanks to AI. In 2024, a Hong Kong-based employee of the British multinational engineering firm Arup sent 200 million HKD (~25.6 million USD) to fraudsters after joining a video call with deepfaked staff (Magramo, 2024). Cyberattacks can be devastating for businesses; in 2013, Efficient Escrow of California had to shut down after having USD 1.5M stolen from them by hackers (Syscon, n.d.; MIS, 2016; California Department of Justice, 2014). 

Contrary to popular belief, it can be argued that the thinking behind AI has been assisted by ancient philosophical reasoning, with thinkers like Aristotle, Descartes, and Homer contributing to some of the logical frameworks behind it (Ikechukwu, 2025). Buchanan (2005) states that “the beginnings of artificial intelligence are traced to philosophy, fiction, and imagination,” (p. 54) with ancient philosopher Homer writing “of mechanical “tripods” waiting on the gods at dinner” being an early example of fantasies of “mechanical assistants”. The Dartmouth Conference in 1956 is widely considered to be the birth of artificial intelligence, as the term was coined there (Buchanan, 2005; Mijwel, 2015). In recent years, models like ChatGPT and Gemini have brought AI to the masses. Therefore, AI is a longstanding field that has been gaining traction in recent years due to widespread adoption. 

The earliest recorded cyberattack can be attributed to have happened in France in 1834, in which two government bond traders bribed a telegraph operator to add characters to official Government reports being sent over the telegraph to indicate the previous day’s market movement, as they lived in Bordeaux, where information took days to arrive from the exchange in Paris by mail coach (Monroe University, n.d.; Schneier, 2018). In recent years, cybersecurity has become more important, as there are many ways cyberattacks can happen; according to ENISA (European Union Agency for Cybersecurity, 2024), these are the top threats: 

Definitions: 

(Stanham, 2023) 

Additionally, according to CrowdStrike, ML in cybersecurity can either be “Automated threat detection and response” or “Analyst-led operations assisted by machine learning” (Stanham, 2023, sec. 4). The prior can replace human cybersecurity workers to improve accuracy and speed while the latter is more suited for more complex cases which ML models may struggle with.  

Artificial Neural Networks (ANNs) are commonly used in ML algorithms because they enable AIs to learn. ANNs are inspired by brains, which have networks of neurons to transmit information, allowing animals with brains to think. ANNs comprise artificial neurons which add multiple numerical inputs, then pass the sum through a function to produce an output (Han et al., 2018) 

3.2. Cybersecurity 

(Barbosa et al., 2024) from the IMF notes that “the aggregated direct losses from cyber incidents have amounted to almost $28 billion” from 2020 to April 2024. Although this number includes accidental incidents, it highlights the monetary risks at stake. IBM (2025) puts the global average loss from a data breach at $4.44 million and the US average loss at $10.22 million, emphasizing the importance of data security. Hiscox, 2023, as cited in Fotis, 2024 notes that the median cost of a cyberattack in 2023 was as follows: 

Fig. 1. Average cost of cyberattacks by country (USD). Data from Hiscox, 2023, as cited in Fotis, 2024, Figure 2.  

The disparity between IBM and Hiscox’s numbers was likely because IBM used the mean, while Hiscox used the median. Therefore, a few high-profile cases likely increased IBM’s numbers significantly.  

3.3. AI in Cybersecurity 

AI can be used in cybersecurity to save human effort when securing systems. According to Mohamed (2025), AI systems in cybersecurity primarily do 8 things:  

IBM notes that AI-assisted security teams saved on average $1.9 million per data breach compared to non-AI assisted security teams. They also noted that 13% of organizations reported data breaches involving their AI models, suggesting stronger AI guardrails should be in place. Furthermore, IBM notes that 1 in 6 data breaches involved AI-driven tasks, of which 37% was for phishing and 35% was for deepfakes.  

3.4. Deepfakes 

Alanazi et al. (2024) states that face-swap deepfakes are made using autoencoders to transform an image or video into code that captures essential features for a face, then a decoder is used to transform that code into another person’s face. For instance, the autoencoder is used to turn person A’s face into code. Then, their face is decoded using person B’s decoder so that person B’s face appears. It is important to note that both person A and B would have both an encoder and a decoder; it’s just that person A’s decoder and person B’s encoder aren’t used in the final face-swap. The paper also mentions another method for deepfakes, known as GANs (Generative Adversarial Networks). These consist of a generator and a discriminator. The generator creates fake content, and the discriminator tries to detect it. When the discriminator can no longer detect that the content is fake, the process is finished. GANs generally produce higher-quality fakes, while face-swap technology is simpler. 

3.5. Phishing 

There are 4 stages to a phishing attack: planning, attack preparation, attack conducting, and valuables acquisition. 90% of organizations faced phishing attacks in 2019, suggesting that they are a widespread phenomenon with possibly large implications if ignored (Alkhalil et al., 2021). However, in a 2019 survey by Proofpoint, awareness was much lower.

Fig. 2. Percent of respondents to a survey knowing what ‘phishing’ means in different G7 countries. Data from Proofpoint, 2019, as cited by Alkhalil et al., 2024 

4. Body 

4.1. Financial impact of cyberattacks 

Cyberattacks can have a large, potentially devastating, impact on organizations. In the US, the mean cost of a data breach was USD 10.22M while the median cost of a cyberattack there was USD 20,000. As said in the literature review, the disparity between the mean and the median could mean that there were some extremes. According to Bennett (2025) at Vena, the average small American business earns USD 1,221,884 of revenue annually. The profit margins for business sectors in the US vary, but the database compiled by Damodaran (2025) at NYU states that the average American firm has a net profit margin of 8.67%. Although profits vary from small to large businesses, using this data, the monthly profits of the average small American business are (very) approximately USD 8,828.11. Since the median cost of a cyberattack in the US was USD 20,000, a cyberattack could cost a small business months of profits. Additionally, since the real median personal income in the US was USD 45,120 annually in 2024 (FRED, 2024), a cyberattack on an individual could be catastrophic to their finances. 

4.2. How deepfakes amplify phishing threats 

Deepfakes amplify phishing threats by increasing how much victims trust fraudsters. Unlike regular phishing attacks that consist of poorly crafted messages, deepfaked phishing attacks are more sophisticated. By seeing replicas of people they confide in, such as family members or colleagues, the victims are more likely to send money over. This is because, as (Campanella et al., 2007) communicates, humans tend to rely on visual and auditory information. Moreover, familiar faces and voices increase the trust victims place in fraudsters (Stride Creative Group, 2015). This stresses the importance of guardrails to be in place to detect such threats before they lead to monetary loss, but some can bypass facial and audio verification systems. As mentioned in the literature review, GANs (Generative Adversarial Networks) are used to make deepfakes that bypass detection, so they can sneak past systems designed to hold back sensitive data. 

4.3. How can AI be used on the defensive frontier 

The statement ‘fight fire with fire’ invokes strong opinions, and may be what cybersecurity needs, where the figurative ‘fire’ is AI. Offensive AI in cybersecurity has wreaked havoc through cyberattacks, and AI is needed to defend against it. Defensive AIs in cybersecurity do many things which standard algorithms do too. However, AI evolves, meaning it can keep up with threats which evolve too.  

AI, as outlined by Singh et al. (2024), can be useful to detect deepfakes, as it can scan individual as well as groups of frames in videos for irregularities. It can also check the audio and cross-check with real data. Moreover, according to (IBM, 2025), security teams using AI thoroughly saved on average USD 1.9M per breach as well. If AI-powered cyberdefense was used in the Arup case, the deepfakes may have been detected and the money may not have been sent over.  

4.4. Body P4: Challenges 

Using AI in cybersecurity systems to defend against deepfake-based phishing attacks entails various challenges. 

Firstly, as mentioned in the literature review, GANs (Generative Adversarial Networks) are trained to be undetectable by the cyberattacker’s own systems (Alanazi et al., 2024). Hence, to detect them, the detectors used on the defensive side must outclass those used on the offensive side. This could lead to both sides innovating themselves into a stalemate as they try to outclass each other. 

Additionally, since recognition for phishing in countries like Japan is quite low (Proofpoint, 2019, as cited by Alkhalil et al., 2024), many individuals and possibly organizations may be unable to detect phishing attacks. Despite countries like Japan with higher average ages having less recognition for the term ‘phishing,’ younger people are generally more susceptible to phishing attacks due to greater impulsivity and trust (Williams et al., 2018, as cited by Alkhalil et al., 2024). One study found that engineering and IT students were the hardest to phish (Bailey et al., 2008, as cited by Alkhalil et al., 2024), proving that those acquainted with disciplines related to cybersecurity were able to detect phishing attacks.  

5. Rebuttals 

There are many rebuttals one can make against the use of AI in defending against deepfake-based phishing attacks.  

Some may argue that the costs of implementing AI in cybersecurity systems are higher than the monetary benefits they bring. However, the various direct and indirect costs can add up to a significant number. An example of this was in 2019, when a Norwegian aluminum maker called Norsk Hydro ASA with ~35,000 employees in 40 countries fell victim to ransomware. Interruption of production alone led to a ~USD 40-50M loss, while getting all software and hardware back to normal cost further millions. In addition to these, there were indirect costs like erosion of confidence in the company and harm to their reputation. After the attack, the company improved their cyberdefense (Fotis, 2024). Furthermore, according to IBM, 2025, the average cost of a data breach is USD 4.44M, while for organizations that didn’t have AI-enhanced cyberdefense and automation it was USD 5.52M and for organizations which used AI-enhanced cyberdefense and automation extensively it was USD 3.62M.  

Others may claim AI is too unreliable to be used in cyberdefense. However, AI has been shown to reduce data breach costs, and IBM recommends the usage of AI in cybersecurity. Moreover, CrowdStrike, an established cybersecurity company, uses AI in their cybersecurity solutions, thus proving that AI is not too unreliable for cyberdefense. 

6. Conclusion 

This paper aimed to investigate the use of AI in defending against deepfake-based phishing attacks and found that its use is important to combat the growing number of AI-powered deepfakes, but it should be accompanied by humans and traditional algorithms to ensure accuracy. Despite it seeming counterintuive to pit one improving system against another (defensive vs offensive AI), there will be an evolving offense and hence the defense must evolve to be ahead. If more organizations were to adopt AI in their cybersecurity solutions, a decrease in data breaches caused by cyberattacks would likely follow. This paper had no experiment and was consequently limited to using data from external studies. However, these external studies were very reputable, with names like IBM and Hiscox. 

7. Bibliography