Customer estimates of electricity reliability

Brent Taylor & Marianne Lourey, KPMG

Notwithstanding recent events first world electricity supply networks generally have very high reliability standards.  So high indeed that in most cases incremental improvements to the network are very expensive.  In such circumstances it is critical for network planners and utility managers to have objective measures of how good is good enough with respect to the reliability of their electricity supply networks.  While planners can take an abstract view on appropriate reliability measures, ultimately that decision needs to be informed by the market – in other words, by consumers.

A recent paper by McKinsey’s[1] outlined a case for not improving reliability of the residential power supply because ‘customers are largely content with their service’ and in any event ‘more than half didn’t know the total length of the outages experienced’.

This is cold comfort for network planners because they know that there are also significant numbers of people that are not content with the reliability of their electricity supply.  In the absence of defensible criteria, they are pushed in different directions by customer complaints, politics, company economics and regulators[2].

KPMG has recently undertaken two projects to evaluate the relationship between the measured reliability of the electricity supply network and customers’ perceptions of that reliability.  Both of these projects revealed defensible reliability thresholds for network planners to base decisions on.

While we agree with much of what was stated in the McKinsey paper, we also disagree on a number of crucial points.  Most particularly we have found that:

n       While most customers are happy with their electricity service, a substantial minority of customers are dissatisfied;

n       Customers can provide reliable and valid estimates of their own supply performance;

n       Algorithms can be developed that convert customer perceptions into reliable and valid estimates of network performance; and

n       There is a measurable reliability threshold above which customers are generally not satisfied with their electricity supply and below which customers are generally satisfied.

In this paper, we discuss the major points of difference between KPMG’s findings and McKinsey’s study (quoted below in italics).

Quote 1: ‘Our recent survey of one electricity distributor’s customers, for instance, showed them to be largely content with their service.’

Our work supports the finding that a substantial majority of customers are largely content with their electricity supply service.  Most customers in our studies have a reliable electricity supply and rate it as so.  The problem with averages, however, is that the outlier responses can become masked.  This is particularly so for highly skewed distributions such as those seen in typical plots of feeder reliability (see example below).

While most customers are largely satisfied with their electricity supply, it is also true that a significant minority of customers do not have what they would consider to be a reliable electricity supply and are not happy with that level of reliability.

Quote 2:’In a comparison of two regions, one with much more reliable service than the other, the responses of customers in each showed that they had nearly identical perceptions of the reliability of their electrical supply.’

The initial analysis in our first study replicated this confounding result.  Initially we could not find a relationship between perceived reliability and actual reliability even though our experience in perceptual mapping suggested that such a relationship should exist.

Part of the solution involved allowing for the fact that consumers experience electricity at a different place in the network to where performance is measured.  Usually the reliability of electricity systems is measured in the high voltage (HV) network while consumers experience supply in the low voltage (LV) network.

Fundamentally, HV network performance and customer service experience is not the same thing.  Customers will experience the same or worse power than that measured in the HV network.

When a customer in a ‘good’ reliability electricity area says that they have poor reliability power, the customer might be right!!  There are many examples of customers experiencing freak distortions in their power supply.

Even if humans behaved as instruments, the measure of network reliability can not be the same as that measured by the customer because local service aberrations in the LV network add ‘error’ to the measurement.  In some cases this error is extreme.  Even a small proportion of extreme variations will destroy any relationships between perceptual measures and network measurement unless they can be accounted for.

The issue is how to account for different reliabilities in the LV and HV networks.  In our experience, if this is not recognised and dealt with, any efforts to calculate relationships between (HV) network measures and perceptual (LV) measures of service will fail.

Quote 3: ‘More than half didn’t know the total length of the outages experienced during the previous year—approximately two-and-a-half hours on average—and more than 80 percent of the remainder significantly underestimated their duration.’

Humans are notoriously poor at exactly replicating the measures of technical instruments, be it time, distance, frequency or electricity reliability.  But as a group, humans can be relied upon to give reliable estimates of some physical events if they are asked in a systematic manner.  Those estimates tend to diverge from the ‘instrument’ measures in a systematic non-linear manner.

The fields of psychology of perception, cognitive ergonomics and perceptual mapping are full of examples where systematic non-linear relationships have been found between instrument measurement and perception.  Electricity network performance is a class of events where we would expect to find mathematical relationships between perception and reality – as indeed we have.

It is also typical of humans to underestimate physical reality.  One only need think of the famous tee shirt design ‘A New Yorker’s view of the world’ in which the features of New York appear close and in full detail while the rest of the world becomes compressed in the background.

This problem can be overcome by designing the perceptual measures in a way that customers can reasonably respond to, and then calibrating these responses against the physical measures.  If this process is carried out rigorously, then the result takes the form of the curve below.  The plot is also defined by a mathematical formula for the relationship.



This curve shows a systematic relationship between feeder measures and customer perceptions.  The line does not pass through the origin of the chart because there is a mismatch between the HV feeder measures and the LV experience of customers.

Quote 4: ‘returns on reliability investments (measured in minutes of improved reliability per euro spent) necessarily diminish beyond a certain threshold, which most distributors have already passed.’

We agree that there are diminishing returns on reliability investments.  There is little or no benefit in improving the reliability of the electricity supply for those people with better reliability than the threshold.  However our view is that important gains can be made by improving reliability for the proportion of people above the reliability threshold. 

It is therefore critical for distributors to establish where their reliability threshold is before making investment decisions.

The example below gives an example of a threshold set by customers.  Typically there is a distribution of reliability in any network.  This distribution tends to be skewed, most customers have reliable power and a minority have unreliable power.



While we are not prepared to generalise about the whole electricity industry on the basis of two studies, we suspect that it is unlikely that ‘most’ distributors will have all customers supplied at a reliability better than their threshold.

Our two studies show that there are substantial differences between jurisdictions both in the relationship of perceived reliability to network reliability, and their reliability thresholds.  In neither case, had the networks achieved reliabilities that exceeded their threshold levels.

In summary

n       Customer perceptions of electricity reliability can be established;

n       Those perceptual measures can be calibrated against network performance;

n       Most consumers are happy with their supply;

n       There is a readily identifiable minority of customers above a reliability threshold that are not happy;

n       Thresholds can be identified and translated into meaningful network measures; and

n       Thresholds are different for each jurisdiction.

Further information:

Marianne Lourey 61-3-9288 6784


Brent Taylor 61-3-9497 5504

[1] Richard Hunter, Ronen Melnik, Leonardo Senni: What power consumers want, McKinsey Quarterly Number 3 2003 (using data for the UK’s Office of Electricity Regulation)

[2] As an example, the Victorian electricity distributors are required to reduce the total minutes off supply over the period 2000 – 2005 – AGL by 15%, CitiPower by 31%, Powercor by 17%, TXU by 15% and United Energy by 37%.  Office of the Regulator-General, Electricity Distribution Price Determnation.  Volume I Statement of Purpose and Reasons, September 2000, p 12