An algorithm doesn’t know what it wants
Making a company data driven doesn’t make it anything as it is just going to duplicate the current biases and opinions of the organization.
According to Cathy O’Neil in her book Weapons of Math Destruction [1] a mathematical model is nothing more than a simulation of the world that often models blind spots reflecting the judgement and priorities of its creators.
e.g. a product myopic organization [2] would not become anything else by applying algorithms, artificial intelligence or machine learning. It would only become more myopic.
It’s relevant to ask: why is the organization making the decisions it does? At the MIT System Dynamics Group they are saying:
We believe any individual is making the best possible decision based on the information they have available — James Paine, MIT, paraphrasing [3]
They are pointing to the information available as the key enabler of decisions.
With advanced analytics the organization could reach new information, new mental models, new types of sense-making. It could understand the world differently and make new hypotheses about what leads to progress. It could train models to test its assumptions in order to open up new worlds of untapped wealth — or close them before they do damage.
The motive of the data driven organization should not be to accelerate or make more efficient the existing biases and opinions of the organization (continuing in the same direction only faster and cheaper) ..
The promise is to help the organization make sense of the world in ways it never could before.
To better understand what the world looks like through other lenses — outside existing bubbles (e.g. through context, relationships, environment, culture). To broaden and diversify its field of view. To lower the cost of experimentation so that any wild assumption can be tested (if you are not experimenting to be surprised you are underutilizing your potential for learning). To be more humble, understand the company is not at the center of the customer’s consciousness, but sometimes only a tiny part of it. To help it have impact in a system where it has no control, but can hopefully assert some modest influence.
The goal of any organization should be to use the gift of the computer to reach somewhere they’ve never been, somewhere previously unreachable or even unknowable — not to accelerate in the direction it is already going (unless they are already perfect?).
Sources:
- Weapons of Math Destruction, https://www.penguinrandomhouse.com/books/241363/weapons-of-math-destruction-by-cathy-oneil/
- Marketing Myopia, Theodore Levitt, https://hbr.org/2004/07/marketing-myopia
- System Dynamics: Systems Thinking and Modeling for a Complex World, James Paine, https://www.youtube.com/watch?v=o-Yp8A7BPE8
