At the CAS Annual Meeting session, “AI Snake Oil,” insurance industry veterans discussed uncharted territory — how AI value is simultaneously at hand and uncannily evasive, as the buzz of the possible seems to continually outpace the art of the practical. Sergey Filimonov of Matrisk AI and Max Martinelli of Akur8, two of the foremost experts in AI for P&C and authors of the CAS AI Fast Track, are sounding the alarm on widespread snake oil hype.
Moderator Issac Espinoza prompted active discussion between Filimonov and Martinelli who counterbalanced all the hype with situations where AI has delivered on its promise. AI techniques like Large Language Models (LLM) are accelerating actuarial work today especially for using unstructured data like text information with extraction, summarization, and categorization, which easily process features that can be directly incorporated into existing predictive models. These same approaches can iterate quickly across updated fields and new text collections.
The presenters also laid out a framework for the audience to better understand products that don’t work, and probably never will, unless what makes them unachievable is somehow solved. For example, many AI models are opaque, difficult to explain how they work, hard to reproduce results, have weak attribution, and have correlated failures versus typical independency assumptions.
The recurring theme was the need to overcome obstacles to progress while continuing to advance areas that were already fit for use.
Filimonov introduced the GenAI Capability Ladder and discussed how Large Language Models as a technology are showing leapfrog performance gains across the vendor landscape now on almost a quarter-over-quarter basis. He broke the ladder into three components: single-LLM features, workflows, and agents. The gains across the ladder steps go from summarization and data feature extraction, to processes orchestrated by code, to the ultimate of self-organizing decision trajectories (the “oilier” part today).
Martinelli, Filimonov, and Espinoza all agreed and advised that the more factual the data and the more consistent the decision making, the more ready for market AI is for those use cases. Modern LLMs are so versatile and accurate that they open countless problem spaces. Literally any unstructured data is now accessible as a structural feature.
They showed how modern feature extraction and summarization tools accurately populate information flows and can branch and bound through decision spaces automatically in a data-driven process. Adopting this step as fast as possible improves efficiency and frees up resources — that is a key benefit of AI: tireless and unerring execution on tasks that probably aren’t the best use for a human’s time.
Using prompts and workflows to offload human tasks should allow organizations to add human attention to areas where effective AI is still struggling – ones with open-ended goals and unconstrained budget gates.
The presenters pointed out there is a chance to have outrageous computing fees, only to achieve an unusable hallucination that might cause liability concerns if not overseen by a human in the loop. AI simply is not ready to reason like a human yet (much less like a data-savvy, business-informed, industry-skilled veteran like an actuary).
They also discussed how the world of “seeing is believing” no longer exists. Photorealistic videos of anyone doing anything anywhere anytime can now be rendered on demand by anyone. This new zone of unbelievable evidence stands in stark contrast to the ways we have worked up to now.
To maximize progress while minimizing risk, focus today’s agent use cases on tasks that are high in complexity and value, have proven, strictly defined step-by-step workflows, and carry a low cost of error. The key enabler for these use cases is the agent’s ability to reason through the task sequence.
The punchline – use AI everywhere you know you can today, but where you have the more unknown unknowns, the less AI is needed, at least for now. That’s great advice on enabling authentic AI capabilities into P&C operations with a risk-managed strategy.
Frank Chang, FCAS, and Past-President of the CAS approached the microphone at the end of the session during Q&A – he summed things up well: “Dumb models give dumb answers.” Try not to do that.
Matin Ellingsworth is president at Salt Creek Analytics.
