How do I know? Well it is simple; almost everyone evaluates situations in the world using metrics that do not represent their goals with high-fidelity.
For example, Peter Thiel is a great businessman and smart guy but like everyone else, his metrics are broken. His thesis is that “innovation is dead”.
“If you look outside the computer and the internet, there has been 40 years of stagnation,” said Thiel, who pointed to one of his favorite examples: the dearth of innovation in transportation. ”We are no longer moving faster,” Thiel noted. Transportation speeds, which accelerated across history, peaked with the debut of the Concord in 1976. One decade after 9/11, Thiel says, we are back to the travel speeds of the 1960s.
http://www.forbes.com/sites/nicoleperlroth/2011/09/12/paypal-founders-innovation-is-dead/
Is going faster and faster a good measure of progress? Is there a point where transportation is fast enough? It is clear that there is not technological barrier to having faster planes but society has made it clear that it does not care to invest in that area to gain that extra speed. Maybe other metrics like miles/passenger/(Joule of energy used) is a more relevant metric. Or maybe it is bad too.
Progress != Growth:
Most people associate progress with growth, but GDP growth by itself is not a good long-term goal because it cannot go on forever. If growth is not sustainable then we should not go after it past a certain point. I do not know the right metric to tell how sustainable a unit of GDP growth is, but I do know that a sustainability component is required to fix the metric.
Why this matters a lot
Creating metrics that reflect your goals (as a person, company, country, ..) is important because people and organizations optimize their activity to metrics. If you are a politician who is judged by whether GDP goes up, you will pursue polices that try to increase GDP. If you are a public company that is judged by short-term earnings growth then you will put a lot of energy into optimizing that.
Fixing metrics is simple but hard
Fixing metrics is very hard in practice but it is conceptually simple because the reason for broken metrics is usually easy to identify.
Top three reasons why most metrics are broken:
- The metric is venerable. It used to make sense but the world changed and it is not hi-fi anymore.
- The metric is too simple. The world is complicated and goals are similarly complex. Simple metrics usually leave out important factors. People like simple metrics so they get popular and gain momentum.
- The metric looks for keys under the lamp post…rather than down the street in the dark where you dropped them. This is related to being too simple but complex metrics can also have this failing. Some goals are hard to represent with metrics with high-fidelity. But that does not stop people from creating metrics to measure those goals. Those metrics are usually chosen for convenience rather than fidelity. An imperfect metric is fine as long as people are aware of the problems and use the metric accordingly.
Even after you figure out that your metrics are broken it is really hard to fix them. A hi-fi metric provides real insight into the world and that is always a challenge. You may even conclude in some cases that there is no simple collections of metrics for a given goal. But fixing your metrics (or your understanding of your metrics) is crucial because failure follows a bad metric around diligently.
