Ethical Issues is written by members of the CAS Committee on Professionalism Education (COPE). The column’s intent is to stimulate discussion among CAS members. Therefore, positions are sometimes stated in such a way as to provoke reactions and thoughtful responses on the part of the reader. The opinions expressed by readers and authors are for discussion purposes only and should not be used to prejudge the disposition of any actual case and do not modify published professional standards as they may apply in real-life situations.
Malcolm “Mal” Reynolds, FCAS, MAAA, CSPA, is a pricing actuary who has spent the majority of his career building predictive models at Independent Insurance Company. His company was recently purchased by a much larger company, Alliance Insurance.
After the merger, Alliance consolidated the actuarial teams. Malcolm was asked to meet with Bob Shephard, also an FCAS and head of the newly consolidated actuarial modeling team.
Bob: I am excited to have you join our modeling team. I’ve heard great things about the modeling work you did for Independent, but they never mentioned you were such a snazzy dresser … I love that brown coat!
Mal: Thanks, I think! I was nervous about the acquisition, but after meeting you and the rest of the team, I’m eager to get to work. I always worked alone on the modeling projects at Independent, so working on a team will take some adjustment.
Bob: That’s understandable, but I hope it doesn’t take you too long to adjust. The first project I would like you to work on is getting Tiffany, your Alliance counterpart, up to speed on the models you developed at Independent. I would like to have her review your development documentation to understand your methods and assumptions. We all benefit by following best practices.
Mal: Um, I don’t have a lot of documentation for my models. I was the sole developer and didn’t have the time to formally document everything. A lot of it is in my head or personal notes, but I would be happy to talk to Tiffany about my process.
Bob: I appreciate your willingness. When you say you don’t have a lot of formal documentation, what exactly do you mean?
Mal: Well, I have some Excel files and the Python code I used to run the model. Nothing is in a format that I could easily hand over to someone else without additional work. As I said, I was the only modeler at Independent, so there wasn’t a need for more than that. My code is well-commented and includes all of the distributional checks and model fit statistics.
Bob: As a member of the Alliance modeling team, you’ll need to document to our standards. We require all steps of the process to be well-documented. Our typical documentation for model development is about 150 pages and covers everything from data preparation and decision support to expected performance statistics. We also require all models be peer-reviewed and independently validated before we use them.
Mal: An average of 150 pages!?! How do you get any modeling done? I was successful at Independent because I could build and deploy a predictive model in under a week, but that didn’t include the extensive documentation you’re describing.
Did Mal violate ASOP 41 and ASOP 56 because he did not document his work to a level that another actuary was able to follow?
Yes
A few working files and some code aren’t enough documentation for another actuary to make an objective appraisal. Additionally, any actuary using the model in the future will need to review this information in order to rely on the model.
ASOP 41, Actuarial Communications, Section 3.2 states: “The actuary should complete an actuarial report if the actuary intends the actuarial findings to be relied upon by any intended user . . . the actuary should state the actuarial findings, and identify the methods, procedures, assumptions, and data used by the actuary with sufficient clarity that another actuary qualified in the same practice area could make an objective appraisal of the reasonableness of the actuary’s work as presented in the actuarial report.”
ASOP 56, Modeling, Section 3.7 states: “The actuary should prepare such documentation in a form such that another actuary qualified in the same practice area could assess the reasonableness of the actuary’s work.” The degree of such documentation should be based on the professional judgment of the actuary and may vary with the complexity and purpose of the actuarial services. In addition, the actuary should refer to ASOP 41, Actuarial Communications, Section 3.8, for guidance related to the retention of file material other than that which is to be disclosed under Section 4.
No
Mal saved all of his working files and has well-documented code. Another modeling actuary should be able to read through his notes and code to know what he was doing.
ASOP 41, Actuarial Communications, Section 3.2 states: “An actuarial report may comprise one or several documents. The report may be in several different formats (such as formal documents produced on word processing, presentation or publishing software, e-mail, paper, or websites). Where an actuarial report for a specific intended user comprises multiple documents, the actuary should communicate which documents comprise the report.”
ASOP 56, Modeling, Section 3.7 further states: “The degree of such documentation should be based on the professional judgment of the actuary and may vary with the complexity and purpose of the actuarial services.”
It depends
It depends on how complete the working files and code commentary are. The completeness and quality of the documentation should be based on how easy it is to interpret and follow.
Ethical Issues Poll — Let Us Know What You Think
Based on the Ethical Issues scenario above, please indicate how you would respond to the following questions.
Loading ... Loading ... Loading ... Loading ...
