Statistics can be very helpful in managing risks. They
can also be used to hide major risks from the uninformed. Managers need to see
through any ignorance, error or deception that can easily lurk in the numbers or
graphs.You don’t have to be a
statistician: common sense and simple math are all you really need.
Data must first be analyzed, interpreted and presented
before management can use it.(A diagnostic control function - see MYPR October
1999)Each step requires action that can introduce errors or allow manipulation.
Data are meaningful only in context – if you don’t
know the subject intimately, get some trustworthy help fast!You should always be wary of sources – ask who, what, when, where, how?Have definitions changed? (See MYPR June 1999)Beware if the source is an interested party or is not shown.Make your own rough estimates.Read
the fine print and watch for words like ‘estimated’ or ‘suggests.’
Surveys and polls require special attention.
It is easy to confuse requirements or goals, which are the
voice of the ‘customer’, with variation, which is the voice of the process.
Similarly, signals and noise must be separated: don’t waste your time on the
latter.It is not enough that your
process meets specs – your managers must also know how their input affects
that compliance.
Managers should be
sensitive to the fact that improper goals can easily cause distortion in the
system or the data, especially if there is a strong incentive (positive or
negative) to meet goals. The temptation to misrepresent outcomes can be very
high – even among otherwise ethical people.
Graphs
are useful, but they too can be distorted: if the vertical axis doesn’t begin
at zero, stop right there.If a
formula seems complicated, get an explanation. If you still don’t understand,
assume it is too complex.Simple is
usually better.Even good stats are
rarely a precise account of anything: they are a representation of the truth but
not truth itself.
Ask
questions, check everything and take nothing for granted. It is your duty to
ensure statistics are not misused, especially if the safety of an innocent and
trusting public is at stake.Good
statistics can provide deep insights, but they can never replace the wisdom born
of a healthy degree of skepticism.
Recommended Reading for Risk
Managers
Understanding
Variation: The Key to Managing Chaos by Donald J. Wheeler. Don’t be put off by the
hyperbolic subtitle: this concise little account on interpreting statistical
process information delivers.So
does Misused Statistics (Second
Edition)by Herbert F.
Spirer, Louise Sprirer and A.J. Jaffe. The latter is a treasure of techniques
for seeing who you can trust and who is trying to bamboozle you with numbers.
Both are mercifully short (135 pages and 263 pages respectively) and seasoned
with cultured wit.Both illuminate
the subject brilliantly to help you separate your statistical signals from the
noise.
Mike
Murphy
spent 17 years (78-96) with Transport Canada, his last five as Regional
Director General, Aviation
in Winnipeg. Originally trained as a professional pilot, he is the
author of an internationally acclaimed 500-page report entitled "An
Evaluation of Emergency Response Services at Airports in Canada,"
currently in its second revised edition (August 1999) and is undergoing
a thirdmajor
revision. He is also the
Chairman of the Air Passenger Safety Group (APSG) www.transport2000.ca/APSG
, a Director of Transport
2000 Canada www.transport2000.ca,
a Director of the Ottawa Chapter of Christian Businessmen's
Committee (CBMC) www.cbmc.net and the Secretary General of the Peugeot Club of North
America (PCNA). www.peugeotclub.org
In
his volunteer role at the Air Passenger Safety Group, Murphy has
investigated and found several air safety documents that misused statistics
to make points that had nothing to do with aviation safety. It
requires a bit of detective work, but can be satisfying when the imposter is
finally exposed. He is of the view that if everyone read statistical
reports with a skeptical eye, the authors of such dubious reports might mend
their dubious ways. Challenge and dialogue are integral to good risk
management. Murphy welcomes the challenge of any idea or statistic
posted here.
Managing Your