How Confident Are You?

Many of us have an obsession in life that drives us with a passion. For me that passion is modeling. For Ruby, that passion is keeping the yard free of undesirables. Ducks are not allowed on our pond, because they're too messy. Ruby takes it upon herself to enforce that rule.


My father has been working on measurement technology in the mining industry for as long as I can remember, so that's his passion. During my recent trip to Vancouver, I helped him set up his own blog as a way to express his views about the state of affairs in that field. As you can tell, if you have a read, he's been rather frustrated with the way that self proclaimed geostatisticians continue to dominate this field with questionable practices. They're his messy ducks that just don't belong on the pond. It's not so hard to chase ducks off a small pond...


But it's very annoying when they simply go and land in a larger pond where they're harder to reach!


My father has been chasing his ducks for many years and they're in a really big pond. The gist of the issue is very simple. Consider that when you're given opinion poll data you're typically told, it's within + or - x% 19 times out of 20. I'm not sure how many of you folks have taken university statistics, but if you have, you'll recognize that as the 95% confidence interval. I.e., if you did the poll an infinite number of times, you'd expect 95% of those polls would produce a result that falls within that interval. It answers the question, how confident are you about your measurement? Isn't it nice that we can quantify the confidence of an opinion poll?

So now ask yourself this. If I'm going to go out to determine how much gold is in some patch of the earth that I would like to profitably mine, wouldn't a quantifiable measurement of confidence be a good thing? For example, suppose we'd need to mine 1 tonne of gold to turn a healthy profit, and the measurements all come back indicating that there are an estimated 1.2 tonnes of gold. Building the mine would seem like a fine idea. But if we're also told that it's within + or - 0.6 tonnes 19 times out of 20, we'd suddenly realize there is a significant risk that we might go broke. And of course in the mining industry, do mines ever produce more gold than what was originally estimated? That's like a software project that delivers early! So why is it that we can quantify the confidence of the numbers produced by an opinion poll, but we can't do the same for something that has such an incredible economic impact?

Most folks probably won't remember the Bre-X scandal but I remember it well. I remember my father phoning me one weekend telling me that he'd been given some mining data to analyze and when he assessed the numbers, he was absolutely sure that those numbers could only be produced by someone salting the samples, i.e., sprinkling gold into them to inflate the results. As it turned out, this was the start of the Bre-X scandal. He figured he'd finally chased those darned ducks off the pond!


No longer could they just continue to play games with numbers to produce overly optimistic results without the proper checks and balances to ensure the numbers actually correspond to physical reality in the ground. It's extremely important to test for spatial dependencies, i.e., to ensure that the variations of the measurements over the physical space in which the measurements occur are correlated in a statistically significant way. Taking those steps would be like a pond without ducks and would be so truly satisfying.


But my father's ducks are really ornery compared to Ruby's ducks, and because they're in such a big pond, his ducks head for deeper water and form little groups that try to ignore him. You can imagine that it's quite convenient to be able to play games with your numbers as a way to manipulate investors. This might all not seem terribly relevant to you personally, but ask yourself, where are your retirement savings invested? How confident are you?