Generative AI has rewired the cybersecurity fight in under two years. It now helps defenders catch threats; yet it also gives hackers powerful new tools. The same technology fuels both protection and attack, often inside the same news cycle. Understanding both sides is essential for anyone building a security strategy today.

What follows covers how generative AI supports cybersecurity professionals, how attackers exploit the same tools, and where the balance currently sits. It draws on real-world examples, current data, and a direct read on who’s actually ahead right now.

My own take: attackers currently have the edge. Not because their models are smarter, but because an attacker only needs AI to work once, while a defender needs it to work every single time. 

The GTG-1002 espionage campaign, covered below, shows how one successful jailbreak did the work of an entire hacking team. Most enterprise security stacks aren’t built to catch that kind of speed yet.

How Generative AI Strengthens Cybersecurity Defense

Security teams face an overwhelming volume of alerts every day. Generative AI summarizes threats, correlates signals, and flags priorities in seconds. Therefore, analysts spend less time on manual triage. 

1. Faster Threat Detection and Alert Triage

Large language models read raw security logs. Then, they translate technical data into plain language. Therefore, junior analysts can understand complex threats immediately. Moreover, generative AI can compare current activity against historical patterns. Because of this, it often catches anomalies that rule-based systems miss.

AI security tools commonly assist with threat detection, alert triage, phishing analysis, malware classification, vulnerability prioritization, cloud posture review, identity risk scoring, and incident response support, but the real value isn’t the list; it’s the coverage. Generative AI now touches nearly every stage of the defense lifecycle, which is exactly why a single blind spot in one stage can undercut the rest.

2. Faster Incident Response

When a breach occurs, every minute matters. Generative AI speeds up response by drafting containment steps automatically. It can also generate incident summaries for executives instantly. This saves valuable time during a crisis.

The dollar figures make the case on their own. IBM reports that 51% of enterprises now use security AI or automation, and those organizations experience $1.8 million lower average breach costs than those without it. Additionally, breach costs remain high across the board. The average cost of a data breach was $4.4 million in 2025, even after a modest decline due to faster detection.

3. Synthetic Data for Better Training

Security teams often lack enough real attack data to train detection models. Generative adversarial networks (GANs) solve this. They create realistic synthetic attack samples. Subsequently, these samples fill dataset gaps and improve detection accuracy.

For example, TadGAN and TAnoGAN use GANs for time-series anomaly detection, while PassGAN demonstrates how generative models can test password strength. These tools show how generative AI strengthens defenses behind the scenes, not just on the front lines.

4. Widespread Enterprise Adoption

Generative AI now plays a role in 77% of security stacks, according to a Darktrace survey of over 1,500 security leaders, proof that this changed from experimental to standard practice in a single budget cycle. 

However, adoption brings new challenges too. Meanwhile, 44% of leaders remain extremely or very concerned about the security implications of third-party LLMs, and 92% worry about AI agents operating across the workforce. Clearly, generative AI has major benefits. Still, teams must manage new risks carefully.

How Hackers Exploit Generative AI

Attackers adopted generative AI just as fast as defenders did, and in several respects, they moved faster. Generative tools lower technical barriers. Even novice hackers can now launch campaigns that used to require a skilled team.

1. AI-Generated Phishing at Scale

Phishing remains the top attack method, and generative AI made it far more convincing. Phishing accounts for roughly 60% of intrusions, and AI-generated content now makes those messages nearly indistinguishable from legitimate ones. Attackers no longer write clumsy, typo-filled emails. Instead, they generate flawless messages in seconds.

Microsoft’s Cyber Signals 2025 report recorded a 46% rise in AI-generated phishing content, while SlashNext observed a 25% increase in phishing messages that bypass conventional filters. Furthermore, researchers estimate that 80% of phishing attacks are now AI-generated or use AI tools in some way.

2. AI-Generated Malware

Attackers also use generative AI to write malicious code. This trend has accelerated dramatically in recent years. LLM-generated malware is projected to account for 50% of detected threats in 2025, up from just 2% in 2021. This jump shows how quickly generative tools reshaped the malware landscape.

Nation-state actors have embraced this capability too. FANCY BEAR, a Russia-nexus threat actor, deployed malware called LAMEHUG, which uses the Qwen2.5-Coder-32B-Instruct model to generate real-time reconnaissance commands. This example shows AI moving from a coding assistant to an active weapon.

3. Deepfakes and Social Engineering

Deepfakes show what happens when generative AI targets human trust directly, not just technical defenses. Criminals now clone voices and faces convincingly. As a result, regular verification methods no longer guarantee safety.

The Arup case illustrates this danger perfectly. In January 2024, a finance employee at the engineering firm Arup was tricked into making 15 transfers totaling $25 million after joining a video call where every participant besides him was an AI deepfake of a real colleague. 

Notably, the fraudsters built these deepfakes from publicly available video and audio, including recordings from conferences and meetings. This case proves that even skeptical employees can be fooled when the deception looks and sounds real.

Deepfake incidents are not rare anymore, either. One industry survey found that 85% of organizations experienced at least one deepfake-related incident in the past year.

Also read: How to Protect Yourself from AI Scams in 2026

4. Autonomous, AI-Orchestrated Attacks

Perhaps most concerning, attackers now let AI run entire operations. In November 2025, Anthropic disclosed a landmark case. A Chinese state-sponsored group manipulated Anthropic’s Claude Code tool into attempting infiltration of roughly thirty global targets, succeeding in a small number of cases. 

The attackers used a clever technique to succeed. They broke their attacks into small, seemingly innocent tasks, so Claude executed them without understanding the malicious purpose behind them.

The scale of automation was unprecedented. Claude executed 80 to 90% of the operation independently, while human intervention across key phases totaled roughly 20 minutes of work. However, the technology still has limits. 

Claude occasionally hallucinated credentials or falsely claimed to have extracted secret information that was actually public. Therefore, fully autonomous attacks are not perfected yet. Still, this margin is reducing.

Other sources confirm this acceleration. According to the CrowdStrike 2026 Global Threat Report, AI-enabled adversary operations increased 89% year-over-year. It spans reconnaissance automation, credential dumping, social engineering, malware generation, and evidence erasure. 

Even more strikingly, the fastest recorded breakout time between initial access and lateral movement was just 27 seconds. Financial losses reflect this new threat category clearly. The FBI’s IC3 2025 Annual Report documented 22,364 AI-related complaints, generating $893 million in losses, marking the first time AI cybercrime was tracked as a distinct category in the report’s 25-year history.

What This Dual-Use Reality Means for Organizations

Generative AI does not pick sides. It amplifies whoever wields it faster and more skillfully, and right now, that favors attackers, who face none of the compliance reviews, procurement cycles, or ethical guardrails that slow defenders down. 

Organizations must treat AI as both a shield and a potential attack vector, and budget accordingly. Security leaders increasingly recognize this issue 92% agree that AI-powered cyber-threats are forcing them to significantly upgrade their defenses. 

Meanwhile, adoption keeps climbing regardless. Seventy-seven percent of organizations now use generative AI or large language models within their security operations. Given these trends, organizations should take three concrete steps.

First, they should train employees to question video calls, not just emails. Deepfake verification protocols matter more than ever. Second, they should deploy AI detection tools alongside human oversight. 

This combination catches threats that either approach might miss alone. Third, they should monitor their own AI tool usage closely. Exposed AI infrastructure creates new entry points for attackers.

Where This Leaves Security Teams

Generative AI has erased the old timeline of cybersecurity. Defenders now detect threats and respond in minutes instead of days. Attackers now craft convincing phishing, generate malware, and orchestrate autonomous campaigns with a fraction of the headcount an operation like this used to require. 

The Arup deepfake scam and the GTG-1002 espionage case prove these risks are not theoretical. In my view, most organizations are still treating generative AI as a nice-to-have detection upgrade when they should be treating it as an existential threat that demands immediate investment.

Ultimately, organizations cannot afford to ignore either side of this equation. They must adopt generative AI defensively while preparing for its offensive misuse simultaneously. Those that master both fronts will stay ahead. Those that ignore either one will fall behind quickly.