At the CAS Spring Meeting in Toronto, members of Guy Carpenter’s Cyber Analytics Center of Excellence (COE) discussed the multifaceted cyber threat posed by artificial intelligence (AI). The presentation included insights from a series of papers they developed on cyber aggregation risks posed by AI and the impact on industry catastrophe (cat) models.
One thesis explored during the session was that despite presenting a multitude of novel risks up, down, and across the technology supply chain, AI alone does not create the types of single point of failure[1] (SPoF) that typically concern catastrophe models. At the same time, AI is likely to increase the frequency and severity of cyber events in ways insurers should register and address — regardless of whether they even write cyber.
AI rarely says, “I don’t know.”
Despite its breakneck evolution and broad adoption, AI is at its heart a fragile technology like many others — in some ways, it’s even more fragile due to its intricacy.
“There are a lot of recent developments around AI and these are very exciting, but we should all recognize that these developments are built on technology that has been evolving for more than 70 years,” said Jess Fung, Guy Carpenter’s head of cyber analytics. Fung pointed to a recent half-marathon of AI-powered humanoids in Beijing — where many of the robots needed humans to run beside them, change their batteries, and pick them up when they toppled — as a whimsical example of AI’s present standing in the march of progress.
While some humanoid runners eventually got up and crossed the finish line, Fung pointed to the irreparable impact caused by some AI-related mishaps as a novel failure point. Academics such as Tom Johansmeyer have pointed to the reversibility of cyber events as a differentiating factor from other systemic risks that effectively puts a lower ceiling on maximum possible losses.
ChatGPT and other large language models (LLMs) may push reversibility to a breaking point. Fung cited the recent example of GDPR-related litigation related to an instance where ChatGPT reportedly hallucinated that a Norwegian man had murdered his own children while the chatbot recited a series of otherwise banal facts about the user.
“OpenAI apparently cannot even erase the incorrect data,” Fung said when describing the incident. “They can only block it from showing up when certain prompts would lead that information to come up. The incorrect information still exists in OpenAI’s dataset for training future versions of ChatGPT.”
Matthew Berninger, principal cyber analyst at Marsh McLennan Cyber Risk Intelligence Center, observed how AI may be fraught with peril even when providing factually correct information.
“In the past, if somebody accidentally gave you access to a health care document, you might not even know about it,” Berninger said, discussing workplace productivity tools such as Copilot. “However, if you ask AI about a certain health condition, it may tell you that your coworker has a similar condition. AI has the potential to inherit and expose security entitlements that are inappropriate.”
“The machine will explore every nook and cranny of your organization and in some cases expose them, for better or for worse.”
Could start to look like an aggregation
While doxing is potentially expensive and traumatizing, it is unlikely to be systemic. AI also cannot in and of itself perpetrate mass incursion, but it has the potential to empower many more individual attacks.
“Part of my background was offensive work,” said Berninger. “When ‘hacking’ into an organization, I often had to solve little puzzles along the way — technologies I may not know, new coding languages. I had a laptop next to me to search commands to do different things.”
LLMs such as ChatGPT can accelerate such learning. “LLMs are not an aggregation in the sense that they affect a thousand organizations at once,” Berninger said. “But if an attacker who could once execute 10 attacks per week can now execute 100, that could start to look like an aggregation.”
AI can also help exacerbate damages once an attacker has “secured a beachhead.” From a social engineering perspective, Fung provided the example of a finance worker in Hong Kong who reportedly paid out $25 million to attackers after they created a deepfake video conference call posing as the company’s chief financial officer and several other staff members. Berninger described this as “better lures and phishing” enabled by AI.
Rich McCauley, senior cat modeling advisor for Guy Carpenter, pointed to AI-enabled polymorphic malware[2] as another potential game-changer for attackers. “One of the trickiest parts of attacks is getting the data out, trying to hide that, and camouflage that in more normal looking transmissions,” he explained. “Dwelling on a system for longer allows for a greater collection of data and can really expand an attack.”
McCauley added that LLMs could also help attackers optimize time spent in a company’s network, noting that in the 2017 Equifax breach, which was one of the largest in history, only 265 out of 9,000 queries returned personally identifiable information (PII). “AI could potentially use pattern recognition to identify more relevant data very quickly,” he said.
You just brought that bug into your house
Systemic risks may also reside beneath the surface or adjacent to widely used AI tools. Berninger compares LLMs such as ChatGPT to a store.
“You’re either leaving your house to go the store or having the store send something to your house,” he says. “Either way, there’s risk.”
For the many organizations relying on externally hosted LLMs to power their websites, Berninger explained “if your operations are contingent on going to the store and it is temporarily closed, then you can’t get the things you need. So, there may be some aggregation risk from ChatGPT going down,” as it reportedly did in 2024 following an attack by a hacker group.
On the other hand, “if you take something like [Meta] Llama off GitHub, and that software has a bug or a backdoor in it, then you just brought that in your house,” Berninger added. In these regards, AI assumes some of the same risks as standard software supply chain.
To the extent LLMs themselves don’t create systemic risks, the conduct surrounding them may. One petabyte of data was required to train ChatGPT 4. The data used to train AI creates multiple aggregation risks on both sides of the table.
“Now, you are seeing [AI] companies aggregating a bunch of data in one place so they can train models on it. That is a security risk that can verge on an aggregation risk,” Berninger said. “Additionally, if you’re using any kind of third-party hosting solution, that becomes essentially a data bank that attackers can go after. If the host is breached, then the attacker may be able to access many different companies’ data — and ransom all those companies.”
He points to the 2024 Snowflake breach, which affected at least 100 Snowflake customers, as one example that gives a glimpse into the risks of migrating data into the cloud en masse.
Whole new avenue of risk
Insurers who do not (knowingly) insure cyber risk cannot necessarily breathe a sigh of relief. Some of the “silent AI” risks insurers are grappling with affect the directors and officers (D&O) and professional liability (E&O) lines.
“Imagine a company officer asking questions to a model as they make a decision, and down the road investors don’t like the decision,” said McCauley. “Suddenly, you have a very interesting intersection between insurance and AI products.”
Fung also noted the usage of AI in the legal profession. “Software such as CoCounsel can help lawyers conduct research much faster than before,” she explained. “It can search through case law, comb through evidence during discovery phase, review, and redline contracts, compare documents, or even prepare timelines for the lawyer to take a case to court.” Instances of lawyers citing AI-hallucinated cases are reaching epidemic levels, often resulting in sanctions.
AI use also has the potential to create new product and operations liabilities.
“Previously, AI was largely contained in a digital box. The algorithms were largely run by data scientists to solve data science problems,” Berninger said. “AI wasn’t telling people how they should eat or behave or think about issues with their family. Now, we have a layer where AI is interacting in very personal way with people — and that opens up a whole new avenue of risk.”
This has already been observed with “death by GPS,” where the AI-powered technology specifically navigates drivers into oceans or over cliffs or more generally rots drivers’ innate navigational and reasoning abilities over longer-term periods.
McCauley points to a potential need for affirmative coverage for AI-related risk, which has recently become more available in the marketplace.
“Existing policy wordings may not fully address losses that come from AI interactions, and sometimes when carriers start getting concerned, they develop exclusions,” he said. McCauley observed that sometimes exposures aren’t appreciated until an event sheds light on them.
“We saw this last year with the 2024 CrowdStrike outage [which was not AI-related], where cyber insurance covered business interruptions related to non-malicious attacks,” he said. “We don’t know how AI will play out, but insurers should be understanding and flexible in their wordings.”
Amid all the uncertainty, it is reassuring that one point seems broadly agreed upon: Despite its many-tentacled nature, AI has not risen to the level of a SPoF.
In developing its research, the Guy Carpenter team communicated with both leading cyber catastrophe models, CyberCube and Cyence.
“Both agreed that the initial impacts are more in frequency and severity, in efficiency, in adding to existing attacks,” McCauley said. “But AI is not implemented in a way that its footprint is broad enough, or it impacts organizations’ revenue by enough, to be considered impactful as an SPoF. It is good to see agreement between model vendors who often have quite different views.”
However, this is likely cold comfort to insurers who may be sitting on mountains of silent AI risk.
Jim Weiss, FCAS, is divisional chief risk officer for commercial and executive at Crum & Forster and is editor in chief for Actuarial Review.
[1] CyberCube defines SPoF as a cyber incident on a shared technology that may disrupt the business operations of a large swath of companies.
[2] SentinelOne defines polymorphic malware is malicious software that can morph its code, making it difficult for traditional antivirus solutions to detect.
