In a company environment, implementing actions based on accurate predictions creates opportunities for generating and maintaining competitive advantage. It can become a key differentiating factor, propelling you to succeed over your competitors.
Human beings have been on a journey to perfect their prediction strategies for hundreds of years, but invariably we don’t get it right.
Weather prediction for example, presents an interesting set of challenges i.e. dynamic changes over time based on a multitude of changing variables. Weather forecasts are made by collecting quantitative data about the current state of the atmosphere, land and ocean. However, due to the chaotic nature of the atmosphere, forecasts became less and less accurate as the difference between the current time and the time of the forecasts increases. Despite significant technological advancement enabling increased computing power, today’s three-day forecasts are as precise as the two-day forecast ten years ago.
We’ve learnt that predictions are not only about data and computational power. Instead, it is our concept of prediction that requires revision.
The trouble with prediction based on history
Almost all forms of prediction use the past as an effective basis for predicting the future. The certainty of the past makes it tempting for forming a basis for extrapolating the future, but this is not the case. Consider two notable quotes illustrating the point:
“There is not the slightest indication that nuclear energy will ever be obtainable. It would mean that the atom would have to be shattered at will.” — Albert Einstein, 1932
“Heavier-than-air flying machines are impossible.” — Lord Kelvin, British mathematician and physicist, president of the British Royal Society, 1895
There are a number of very human reasons why a view of the past isn’t a reliable yardstick for the future. Consider your ability to remember details about events as you move through life. The process of aging comes with some real-life side effects like memory loss. Even your cultural biases change over time. People form a conceptual or inaccurate view of the past, and thus a conceptual, inaccurate definition of the future gets created.
Secondly, if institutions requiring accurate predictions can’t rely on humans, can they rely on humans with tools that take the long view? In the latter case, predictions are made by humans using data and statistical methods. Unfortunately, this too results in data getting frozen in time, making inaccurate predictions that have real ramifications for the people utilising these systems (credit scoring, for example).
Prediction in real time
Calibration is the process of verifying the accuracy of measuring instruments by comparing readings to a known standard. In a technical setting, continuously calibrating your measuring instruments is necessary due to factors like wear and tear, environmental conditions, or aging. Regular recalibration helps maintain the instrument’s accuracy and reliability over time.
In a world where events don’t occur as a result of a linear effect, but rather through a process of emergence from a complex network, where innovation is dynamic and rapidly evolving, we need predictive models that don’t use history as their reference point.
For prediction to be truly intelligent, it should function in real-time. This approach allows artificial intelligence to predict which machine learning model is most appropriate for a given situation, allowing you to recalibrate predictions continuously. By using models dynamically, with AI to automate the process, your predictions account for changes as humans evolve over time.
