What is customer outcome testing and why is it important?

Following our last Day 2 series article about Monitoring Outcomes, it is important to consider what role customer outcome testing plays in enriching the overall monitoring that firms are required to do.

What is Customer Outcomes Testing?

Outcome testing is the holistic assessment of whether an individual customer has received a good outcome, by looking at the sequence of activity through the customer journey. This goes beyond merely the testing of adherence to process and policy, but can identify scenarios when the combination of interactions, processes and policies may give rise to the risk or potential risk of poor outcomes. 

This assessment requires firms to review the customer journey, identify the key risks and actively assess those risks, as well as processes and policies to determine whether the customer has received value for money, good quality communication and customer service, and has ultimately purchased or used a product which met their needs. Outcome testing should be carried out routinely to assess whether the right outcomes have been achieved across:

1. Parts of a customer journey; and the
2. End to end customer journey.

Assessment of the customer journey can include isolated testing around specific roles, but should also include assessments over agreed periods in time to identify any wider risks. For example, you may look at the sales process for a given product plus three months worth of usage to help identify any risks that might arise. End-to-end customer journey testing is also vital to assess how processes, customer communications and the delivery of customer service interacts with customer behaviour and product usage, to ensure the customer is receiving good outcomes.

Why is it so important?

1. Outcome testing is an essential input to the overall monitoring of outcomes across the firm, as it gives confidence that good outcomes are not just being achieved at a top line level (as is often monitored through good quality MI), but also at an individual customer level. As a result, this type of holistic review allows firms to understand whether there are scenarios that could give rise to poor outcomes either for individual customers or a range of customers in particular circumstances. This detailed insight enables firms to identify actual or potential risks, put them right for individual customers, and through root cause analysis take action to prevent other customers experiencing the same risks.
2. Customer outcome testing is not just a valuable input into the wider MI that monitors outcomes, it is also an important tool for drilling down into potential problems when key risk indicators are out of tolerance. Thematic customer outcomes testing has a valuable role in the second line, enabling the business to investigate areas of inherently higher risk, and to reactively test customer outcomes when routine MI monitoring suggests there are risks. The FCA is expecting that where firms identify risks within their monitoring, that action is taken to explore those risks, isolate their root cause and take steps to prevent or mitigate future risk to other customers.
3. The FCA is keen that over time firms invest in their MI to be able to monitor customer outcomes across many lenses, for example outcomes by distributor, vulnerable vs not vulnerable, different customer personas and locations. For many firms building granularity of MI at this level takes time, and often cooperation from other firms in the distribution chain. It is important that firms use other methods of monitoring, such as customer outcomes testing in this intervening period, to provide a level of comfort that customers are receiving good outcomes across their population.

Overall customer outcomes testing is a critical tool for firms in demonstrating compliance with Consumer Duty. It is important to have the right customer outcome testing framework, and methodology for each customer journey. It is also important to determine what level of sampling is required to deliver meaningful results across customer populations, and ensure you have the right capacity and skill within the relevant customer outcome testing teams to carry out this activity.