Internal model validation: adding value and removing complexity

In a recent roundtable, we brought together industry practitioners involved in validation work to share their insights - covering validation within both Lloyd’s and the wider London market.

The topics we discussed included the evolution of validation over the last ten years, stakeholder engagement, quality and efficiency and hot topics for the upcoming validation season.

Is validation evolving enough?

We generally agreed that the validation process had not changed significantly in the years of Solvency II, and that it needed to develop to remain up to date with current requirements.

Certainly from our experience, firms are beginning to ask the question as to whether the validation process is still fit for purpose.

Stakeholder engagement: quantitative versus qualitative outputs 

A key discussion point was about striking the right balance of quantitative information in a validation exercise. Our roundtable attendees noted that there has been an increasing reliance on point estimates over the past ten years. This has two effects: 

1. It leads to anchoring to prior results but does not equip the board with the right information for effective challenge.
2. It increases the danger of distracting senior management from issues that are not material to their organisation and risk profile, as a point estimate does not convey information about its accuracy.

To address these issues, attendees shared strategies that have worked for them including the use of ranges when communicating results to senior management. 

These provide a more realistic picture of the risks faced by the business by allowing for some uncertainty. By providing ranges rather than point estimates, the group emphasised that it gives context for stakeholders to understand and assess the results. It also allows them to temper their response to changes. For example, a change to business plans leading to changes in the solvency capital requirement (SCR) can quickly be assessed for materiality by measuring against the model output ranges used in validation.

Stakeholder interaction was also discussed as an area which needed improvement. One attendee wished for a more effective validation framework, which would require consistent and regular training for non-executive directors and other board members. 

Boards have matured over time - the current members are very different to those from ten years ago and have different areas of experience and expertise. Meanwhile, the models and procedures will also have developed, so there will naturally be a need for regular realignment and training.

Improving validation quality through collaboration

The importance of building partnerships with other teams was highlighted, whether working with first line capital or with the finance team. 

By creating a collaborative environment rather than operating in silos, the quality of the validation outputs can be improved, and a feeling of partnership can be developed. For example, it was shared that successful collaboration with the first line enabled the model development plan to prioritise areas of focus for future validation.

Participants mentioned that they were actively pushing back against validation as a tick box exercise. Instead, it should be viewed as a powerful tool to help the business make better decisions. But it was noted that validation must still satisfy the original requirements: providing comfort to the board, the regulator, and other stakeholders. 

One proposal to unlock more benefits from the validation cycle was to work with the first line team more efficiently. This can be done by transferring some of the initial checks to first line, who are best placed to address these, freeing up validation teams to consider wider areas of focus from an earlier stage.

Key focus areas 

One participant suggested leveraging historical validation to find and focus on areas that will “move the dial” – whether materially impacting the total capital requirement, individual risk areas, or any other important business metrics. 

We carried out a short survey during the roundtable, asking 15 attendees to select the key themes of focus for the upcoming validation cycle. The results are shown in the graph below:

The areas identified were:

• Underwriting – both profitability and prospective loss ratios. This was expected to be an area of particular regulatory focus, given the ‘Dear CEO’ letter from the Prudential Regulation Authority (PRA).
• Geopolitical risks – covering both the areas from the 2023 exercise, as well as any additional areas as the current political climate develops.
• Reserving - especially back year deterioration and how this contrasts with prior reliable reserve releases. 

Efficiencies within the validation process

A repeated concern raised by participants was the constraints (both time and resources) of validation.

Regulatory timelines are getting tighter while the validation work remains increasingly dependent on the work of other teams. Therefore, material changes to their work can have significant knock-on impacts to validation which has implications on the output quality. For instance, triggering an unexpected major model change close to the Lloyd’s Capital Returns (LCR) submission will add a lot of pressure to the validation team. 

The roundtable then moved on to discussing the challenge of creating efficiencies within validation, specifically automated validation. We have our own version of such a tool called validateR.  We noted that a small number of participants had tried down this road but were often distracted by other more pressing concerns.

Automation has many benefits, including allowing the validation team to spend more time providing valuable feedback, focussing on key areas that need more attention and providing the opportunity to increase interaction with other teams, building quicker feedback loops.

Next steps

Our roundtable provided a valuable platform for insurance experts to discuss challenges and opportunities within validation. It was useful to discuss the market observations we have seen, but also to learn more about specific challenges that insurers face and share practices on how to overcome those challenges. 

If you’d like to discuss further, please get in touch with Julien.