Artificial intelligence (AI) is polarizing. It excites the futurist and engenders trepidation in the conservative. In my previous post, I described the different capabilities of both discriminative and generative AI, and sketched a world of opportunities where AI changes the way that insurers and insured would interact. This blog continues the discussion, now investigating the risks of adopting AI and proposes measures for a safe and judicious response to adopting AI.

Risk and limitations of AI

The risk associated with the adoption of AI in insurance can be separated broadly into two categories—technological and usage.

Technological risk—data confidentiality

The chief technological risk is the matter of data confidentiality. AI development has enabled the collection, storage, and processing of information on an unprecedented scale, thereby becoming extremely easy to identify, analyze, and use personal data at low cost without the consent of others. The risk of privacy leakage from interaction with AI technologies is a major source of consumer concern and mistrust.

The advent of generative AI, where the AI manipulates your data to create new content, provides an additional risk to corporate data confidentiality. For example, feeding a generative AI system such as Chat GPT with corporate data to produce a summary of confidential corporate research would mean that a data footprint would be indelibly left on the external cloud server of the AI and accessible to queries from competitors.

Technological risk—security

AI algorithms are the parameters that optimizes the training data that gives the AI its ability to give insights. Should the parameters of an algorithm be leaked, a third party may be able to copy the model, causing economic and intellectual property loss to the owner of the model. Additionally, should the parameters of the AI algorithm model may be modified illegally by a cyber attacker, it will cause the performance deterioration of the AI model and lead to undesirable consequences.

Technological risk—transparency

The black-box characteristic of AI systems, especially generative AI, renders the decision process of AI algorithms hard to understand. Crucially, the insurance sector is a financially regulated industry where the transparency, explainability and auditability of algorithms is of key importance to the regulator.

Usage risk—inaccuracy

The performance of an AI system heavily depends on the data from which it learns. If an AI system is trained on inaccurate, biased, or plagiarized data, it will provide undesirable results even if it is technically well-designed.

Usage risk—abuse

Though an AI system may be operating correctly in its analysis, decision-making, coordination, and other activities, it still has the risk of abuse. The operator use purpose, use method, use range, and so on, could be perverted or deviated, and meant to cause adverse effects. One example of this is facial recognition being used for the illegal tracking of people’s movement.

Usage risk—over-reliance

Over-reliance on AI occurs when users start accepting incorrect AI recommendations—making errors of commission. Users have difficulty determining appropriate levels of trust because they lack awareness of what the AI can do, how well it can perform, or how it works. A corollary to this risk is the weakened skill development of the AI user. For instance, a claims adjuster whose ability to handle new situations, or consider multiple perspectives, is deteriorated or restricted to only cases to which the AI also has access.

Mitigating the AI risks

The risks posed by AI adoption highlights the need to develop a governance approach to mitigate the technical and usage risk that comes from adopting AI.

Human-centric governance

To mitigate the usage risk a three-pronged approach is proposed:

  1. Start with a training program to create mandatory awareness for staff involved in developing, selecting, or using AI tools to ensure alignment with expectations.
  2. Then conduct a vendor assessment scheme to assess robustness of vendor controls and ensure appropriate transparency codified in contracts.
  3. Finally, establish policy enforcement measure to set the norms, roles and accountabilities, approval processes, and maintenance guidelines across AI development lifecycles.

Technology-centric governance

To mitigate the technological risk, the IT governance should be expanded to account for the following:

  1. An expanded data and system taxonomy. This is to ensure the AI model captures data inputs and usage patterns, required validations and testing cycles, and expected outputs. You should host the model on internal servers.
  2. A risk register, to quantify the magnitude of impact, level of vulnerability, and extent of monitoring protocols.
  3. An enlarged analytics and testing strategy to execute testing on a regular basis to monitor risk issues that related to AI system inputs, outputs, and model components.

AI in insurance—Exacting and inevitable

AI’s promise and potential in insurance lies in its ability to derive novel insights from ever larger and more complex actuarial and claims datasets. These datasets, combined with behavioral and ecological data, creates the potential for AI systems querying databases to draw erroneous data inferences, portending to real-world insurance consequences.

Efficient and accurate AI requires fastidious data science. It requires careful curation of knowledge representations in database, decomposition of data matrices to reduce dimensionality, and pre-processing of datasets to mitigate the confounding effects of missing, redundant and outlier data. Insurance AI users must be aware that input data quality limitations have insurance implications, potentially reducing actuarial analytic model accuracy. 

As AI technologies continues to mature and use cases expand, insurers should not shy from the technology. But insurers should contribute their insurance domain expertise to AI technologies development. Their ability to inform input data provenance and ensure data quality will contribute towards a safe and controlled application of AI to the insurance industry.

As you embark on your journey to AI in insurance, explore and create insurance cases. Above all, put in a robust AI governance program.

Find more blogs on AI
Was this article helpful?

More from

Making HTTPS redirects easy with IBM NS1 Connect

3 min read - HTTPS is now the standard for application and website traffic on the internet. Over 85% of websites now use HTTPS by default—it’s to the point where a standard HTTP request now seems suspicious.  This is great for the security of the internet, but it’s a huge pain for the website and application teams that are managing HTTPS records. It was easy to move HTTP records around with a simple URL redirect. HTTPS redirects, on the other hand, require changing the URL…

Announcing Dizzion Desktop as a Service for IBM Virtual Private Cloud (VPC)

2 min read - For more than four years, Dizzion and IBM Cloud® have strategically partnered to deliver incredible digital workspace experiences to our clients. We are excited to announce that Dizzion has expanded their Desktop as a Service (DaaS) offering to now support IBM Cloud Virtual Private Cloud (VPC). Powered by Frame, Dizzion’s cloud-native DaaS platform, clients can now deploy their Windows and Linux® virtual desktops and applications on IBM Cloud VPC and enjoy fast, dynamic, infrastructure provisioning and a true consumption-based model.…

Best practices for augmenting human intelligence with AI

2 min read - Artificial intelligence (AI) should be designed to include and balance human oversight, agency and accountability over decisions across the AI lifecycle. IBM’s first Principle for Trust and Transparency states that the purpose of AI is to augment human intelligence. Augmented human intelligence means that the use of AI enhances human intelligence, rather than operating independently of, or replacing it. All of this implies that AI systems are not to be treated as human beings, but rather viewed as support mechanisms…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters