You Get What You Measure

I recently read Richard Hamming's lecture-essay called “You Get What You Measure”.

Let’s call it YGWYM.

Hamming was a mathematician and coding theorist who worked on the Manhattan Project and at Bell Labs, where he shared an office with Claude Shannon.

His point with YGWYM was to push his students to think further about measurement systems and their impact.

Because "what you measure is what you see, and in turn what conclusion you draw."

One of Hamming’s go-to analogies comes from physicist Alfred Eddington who posed that if an ichthyologist goes out to sea with a net with two inch mesh, throws the net overboard and pulls in a haul of fish, he will conclude things like "all life in the sea is more than 2 inches big”.

The (incorrect) observation is defined by the instrument of data collection. And everything that follows as well.

Another of Hamming’s points in YGWYM is about how accuracy gets confused with relevance.

Hamming pushes against the tendency to measure “hard" things like the number of people in a given political party, rather than “soft” things like political attitudes.

Because people want to measure things that are easily measurable.

Throughout the course of YGWYM Hamming provides examples of accurately counting the wrong things to make the point abundantly clear.

He believed this bias ran deep and wanted his students to put relevance first, every time. No matter how difficult.

The bottom line was don’t confuse what you CAN MEASURE with what you NEED TO KNOW. And that facing up to this may be the challenge of a lifetime.

...

The above image illustrates the concept of "Hamming Distance" a type of metric for error detection/correction in coding theory.