In this installment of Inside Variance, Kate Niswander interviews author and CAS Board Member Jessica (Weng Kah) Leong. Leong and her coauthors Shaun S. Wang and Han Chen are the recipients of the latest Variance Prize for their paper “Back-Testing the ODP Bootstrap of the Paid Chain-Ladder Model with Actual Historical Claims Data.”
Briefly describe your paper.
Have you ever wondered if the common bootstrap model for measuring reserve risk actually works? That is, if you’d used it over the last 30 years for hundreds of companies to identify, say, the 90th percentile of your reserves, then you would have exceeded this number 10% of the time? This paper answers that question. (Hint: No, it doesn’t. Read the paper!)
Why did you choose to write about this particular topic?
The bootstrap model is a very popular way of measuring reserve risk. It helps insurers with important decisions, like how much capital you need to back your reserves. We use it because it’s an elegant mathematical solution, and it is grounded in what we’re familiar with — loss triangles. But no one’s checked to see if it actually works. Considering the wide reliance on the bootstrap model, Shaun, Han and I thought that we should do a rigorous back-testing of the model.
Who is your intended audience?
Actuaries who are estimating reserve risk. I hope they read this and get more insight into the way reserves really behave by looking at data on how reserves have moved in the past. The bootstrap model assumes that reserves move pretty randomly. They don’t. There’s a very real reserve cycle.
What makes this paper unique?
There isn’t much actuarial literature on back-testing models to check their performance on real data. That’s probably because it’s painful and expensive to do it. But it’s important. We produce hundreds of papers with new models and methods, but how will we know if they are advancing actuarial knowledge?
We used the Annual Statement data, and some of this is on the CAS website for free — I think that’s a great start for other budding back-testers.
Was there anything that surprised you during the course of your research?
Yes! Before I wrote this paper, I discovered the reserving cycle. This paper is the first time it’s appeared in a peer-reviewed journal. When I first created it, I was shocked — there was such a regular pattern to the way reserves move. I had to check it with someone to make sure it wasn’t a mistake.
On the x-axis are accident years. The lines show different ultimate loss evaluations across accident years. For example, the flat horizontal line at $1.00 shows that, at 12 months of evaluation, we thought that the ultimate loss was just $1 for each of these accident years. The next red line is at 24 months of evaluation. So, for example, at accident year 2000, we thought the ultimate loss would be $1 at 12 months, and then at 24 months we changed our minds and estimated $1.02 instead. For this accident year, ultimately, at 120 months, we estimated $1.10.
This is created using the booked ultimate loss for seven lines of business, for the whole U.S. P&C industry. It shows what reserve risk really looks like.
What are you working on now?
Around a year ago I took a position in predictive analytics at Zurich Insurance. I love it. My focus is now around business execution of insights. I also think there’s a lot of room for research around predictive analytics for long-tailed
lines.
