ALTOM: How to measure seemingly ‘incalculable’ risks

You should know right off that I’m not an enthusiastic first adopter of technology, as so many technology columnists are. I’ve seen too many ideas and gadgets run straight into reality’s hard, cold, brick walls and disintegrate. Too many have proven to be fun but not productive. Fun is good, but making money is arguably what leads to fun.

No, my goal in life isn’t pushing technology, but applying appropriate technology to workplaces. Every decision about replacing or updating equipment or software has both a cost and risk component. How much does it cost, and what can happen if it doesn’t pan out? The more reliant a company is on its technology, the more risk it has to consider.

The problem is that most of us are actually dismally bad at assessing risk. We tend to think we’re good at evaluating it, but we’re not, so we focus too much on cost. We’re really bad at almost everything that smacks of statistics. People will tell you that foreign aid is 10 percent or 20 percent of the U.S. budget, when it’s really 1 percent. They’ll quote the odds of being the victim of a violent crime at ridiculously high levels, when the probability for most of us is less than 1 percent.

The same is true of business-risk evaluations. Most of them are wildly off the mark. Part of the reason is that most business folk assess risk emotionally and spontaneously, without bothering to think it through. And for most purchases, that’s probably good enough. It’s not worthwhile spending a full day calculating the risk in buying an iPhone versus an Android.

But let’s say you’re supplying smartphones to an entire sales force. The contracts, repairs, replacements and switching costs are all factors that should be shaken and stirred in a spreadsheet before making the deal. What’s the probability that the phone will break just when your sales guy has a hot lead and needs a quote right away?

Such decisions may deserve a risk analysis, but all too often those analyses have run aground on simple risk evaluation. There are times you can’t rely on data, because there isn’t any. The technology is too new, or the vendor won’t share the data. Guessing seems futile. Get three people in a room and you’ll likely get three much different figures. If you can’t nail down a fairly narrow range of risk, how can you do an analysis?

In his book “How to Measure Anything: Finding the Value of Intangibles in Business,” Douglas Hubbard suggests a solution to the problem of risk evaluation. He advocates something he calls “calibration.” People become calibrated when they are able to give realistic guesses for probabilities, mostly in the form of ranges, or what’s called “confidence intervals.”

Hubbard suggests a number of techniques for calibration. One is taking a test that requires you to assign a probability to your answer. For example, one question says, “In what year did Sir Isaac Newton publish the Universal Laws of Gravitation?” The important thing isn’t to figure out when he published (1687, by the way), but to determine how well you can bracket the dates so that you are 90 percent sure your answer is somewhere in there, without being silly about it (we know it was published after 400, for example). His point is that, although you probably have no clue what the exact date is, you do know something about it, and you can assign a reasonable probability to your guesswork range.

His other techniques work similarly, by having you constantly examine your estimates and confront your own bias. Eventually, you get used to keeping your bias out of things and feeling comfortable with ranges, instead of dead-on estimates. One of Hubbard’s major points is that the fetish in many places for precise estimates is counterproductive. If all you know is a range of possible values, and that gets you approximately right, it’s better than being precisely wrong.

Those who work with statistics know instinctively what Hubbard is trying to say. They live in numerical miasma, and feel quite comfortable there. An old joke has two statisticians hunting deer in the woods. One shoots wide left, one shoots wide right, and they high-five shouting, “On average, we hit it!”

Business folk are too often addicted to a cult of precision. Too many decisions in business are taken right from the hip, without the benefit of real risk analysis, because it’s believed the risk can’t be adequately determined, so why bother? Hubbard demonstrates that you don’t have to be a statistician to make good risk estimates. That’s good news when you’re trying to decide whether to bet on new technology. You know more than you think you do.•

__________

Altom is a consultant specializing in pairing businesses with appropriate technology. His column appears every other week. He can be reached at taltom@ibj.com.