2023 Q1 | Edition 1 | Article 1
What hypothesis tests teach us about flawed thinking.
It’s right to fall back on evidence when trying to make judgments. Situations change, often faster than human perception can keep up, and the right data can help us to see the problem before it becomes too late. A perfect example of this is climate change. Sitting in a cold northern climate in January, the argument that the planet is getting hotter is easy to dismiss. It doesn’t feel cold out there. How do we know that we know that the temperature is changing?
Measuring the whole world
Keeping track of all the planet’s land, sky and oceans is no easy task. Metadata collated by Nasa suggests that more than 90% of the observed global warming is not going on down any street you’re likely to be walking down this week, so it’s no wonder we don’t feel the dangers around us. The additional degrees are largely stored in the top few meters of the oceans creating havoc with how heat is transported and distributed to cities around the planet (https://nasa.gov). Sometimes, a simple bit of data can help us to understand an impossibly complex problem. But is that figure of 90% correct? And how high is the rise?
A range of possibilities
When you drill down into the science a little deeper, estimates on the heat gain of the ocean are cited in slightly less certain terms. According to measurements collated by the NCEI Ocean Heat Content, the situation gets a little trickier to measure. According to their data, the ocean averaged heat gain rates of 0.37 (± 0.05) to 0.44 (± 0.12) on data between 0m – 700m in depth 1993-2021 (see https://ncei.noaa.gov). That means that the change in the ocean could be as low as 0.32 and could be as high as 0.64, and that’s just for surface temperature. They may look like small numbers, but in the context of climate change, these small values matter.
We shouldn’t trust statistics blindly. One of the reasons we use data is because humans are susceptible to uniqueness and biases, and we can’t help bringing these to every decision we take. The Halo bias has a long history, first identified in 1977, and describes how people tend to make global evaluations based on individual attributes (Nisbett and Wilson, 1977). In other words, we might know someone in one context, such as their ability in sports and make general positive associations with that person – she’s a great person. We become willingly blind to the complexities of that person and all the other people we meet. This tendency can also transfer to blind trust in statistics. Evidence gathering and complex mathematical analysis comes with a scientific air and a sophisticated approach that clearly seems superior to pure opinion.
That is where the hypothesis test has a lesson for us. When we want to study something as complex as the world’s oceans, land mass and the atmosphere, we obviously can’t put thermometers everywhere. Many areas of the planet are inaccessible, and even if we found a way to monitor temperature, say, across every 100km of the planet, taking a reading say every hour, it would still produce a data set that was too big for current computing to process. Instead, we ask a few experts, use our judgment about where to put the thermometers and how many we need to start making the best judgements. In any data set, the measurements will be a little unpredictable. Temperatures rise and fall dramatically in one day, for example. This is variance.
Understanding what we don’t know
The hypothesis test takes into account the level of variance in the data set, and how many samples we look at and produces a result that assumes that there is a level of error in the result – a confidence interval. It has a lower ‘bound’ a likely minimum, and an upper ‘bound’ a likely maximum That is why the NCEI give us a ‘confidence interval’ or a range of possible temperatures with the ± values.
It's also important to note that the figures do not necessarily refer to the whole ocean. We have the statement that the temperatures were only taken at 0m – 700m in depth, and we need other data to know what was happening below that. In the full academic report, there were details on geography, time and other relevant information. The make of the thermometer used might even be relevant to know. Was it designed to work at those atmospheric pressures?
A parametric test is a reminder that measurements are rarely precise unless the thing we’re trying to learn about is simple. They are a jumping off point for further discussion. We need to avoid halo bias, and one good or bad finding is not necessarily grounds to radically pivot and change.
But there is also a key takeaway from the results shared by the CDEI. Is climate change real? If the lowest figure in the confidence range dipped below zero, we could argue that these findings were inconclusive. The true value of a confidence interval is when we have a target in mind, in this case zero – the point where no change can be observed. If the confidence range crosses that magic line, we have to conclude that we just don’t know enough. Even in the most optimistic value from this study, the case where the lowest possible temperature change recorded was potentially 0.32, it’s still above zero, and this data supports the argument that ocean temperature rise is real. With that knowledge, we can start to ask more questions to understand the problem and fix it.
Our next article …
Being confidently wrong
A confidence range allows you to be wrong, but still more or less in the right place. It’s important to understand when your knowledge is just broad enough or precise enough to reach conclusions
Sources used:
Nisbett, R. E., and T. D. Wilson. 1977. The halo effect: Evidence for unconscious alteration of judgments. Journal of Personality and Social Psychlogy 35 (4): 250-256
For more on confidence intervals, see The National Library of Medicine’s explanation
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5723800/all begins with an idea. Maybe you want to launch a business. Maybe you want to turn a hobby into something more. Or maybe you have a creative project to share with the world. Whatever it is, the way you tell your story online can make all the difference.
