Where are you going?

Measure

What is the use of running when we are not on the right road?

German Proverb

A couple of months ago, I attended a conference on sound and acoustics held at Indian Institute of Science in Bangalore. I didn’t know the first thing about acoustics, but the university was only half an hour’s drive away and my wife was out of town so I had nothing better to do on the weekend and I thought I’d try to learn something new.

During one of the lunch breaks, I was approached by a post graduate student from a small university I had never heard of. He was facing some trouble in his project. He was working on speech enhancement. What this means is that the sound is captured in a noisy or distorting environment (think of speaking into your phone at a busy traffic intersection). This noisy speech is then fed into a computer which runs an algorithm to clean it and produce clear sound at the output.

Variations of this technology are seen in many places. If you have heard any old music tracks which have been digitally remastered, the chances are that some such enhancement algorithms were used. Anyway, it remains an area of active research as new algorithms are developed to deal with new kinds of noises.

This student had been trying various methods in a more or less ad hoc way. His mid-term project review however had gone very badly. The reviewer had asked him why he had chosen the methods he had and why didn’t he try a Wiener filter. Now the kid had no idea what a Wiener filter was and wasn’t sure how to proceed.

I told him I could explain that (I have used Wiener filters on several occasions in my work) but I wanted to learn more about what he was trying to do. For instance, how did he measure how good a method was? Or to put it in another way, if he had 2 ways of cleaning a sound recording how would he decide which was better?

It turned out he just listened to the output and judged one to be better than the other.

I was appalled.

As far as I was concerned, the project hadn’t even begun. The boy had no idea what he was trying to do. His measure was highly subjective. Suppose by some miracle he did strike upon a way of cleaning up noise better than anything on the market, there would be no way for him to know this. Also, anyone reading his report would have no way of telling how good his work was other than the student’s own word for it.

One thing he could have done for instance was to start with a clean sound. Then add some noise to it to reduce the sound quality. Then he could have run his noise cleaning method on this bad sound and check to see how close the output was to the original clean sound he started with.

I explained Wiener filtering to him but I also advised him not to get into all that until he had sorted out the measurement problem.

The key point is that if there is a clear and definite goal, you need a way of telling how close you are to it. To do this, you need the right measure. In this era of big data and analytics, I can’t emphasize enough on the word ‘right’.

The most egregious example of using the wrong measure was in the Vietnam War. The US Secretary of Defence at the time was former Ford boss Robert McNamara. McNamara was a genius with numbers. He and his team had saved Ford Motors from bankruptcy by bringing in the power of management and financial control.

mcnamara

Robert McNamara

In the Vietnam War, McNamara brought in the same analytical thinking. Since the aim of the war wasn’t to win any territory but rather to protect South Vietnam, the army couldn’t use the metric that was most familiar, the extent of enemy area captured which was so familiar to civilians from World War 2 films. Instead the aim was to break the VietCong insurgency which. So they resorted to a Search and Destroy strategy whose success they chose to measure with a “body count” relying on the idea of attrition warfare to wear out the North Vietnamese forces.

There was tremendous pressure on individual units to produce a high “body count”. This resulted in fraud in a massive scale. Even the Defence Department believed that the figures needed to be deflated by almost a third. Even so, this single minded focus ignored every aspect of counter insurgency such as creating safe zones for civilians and establishing relations within the community. The “body count” metric became notorious in the US military among Vietnam veterans.

Regrettably this metric is still being used. One ongoing example is President Duterte’s attempt to crush drug trafficking in the Philippines through a campaign of state sponsored violence and killing.

Another example around the same time of how bad measurement can destroy the mission was in the 1960s when Japanese companies entered the US market. American executives were baffled by how eager the Japanese were to sell their produce even at substantial losses to themselves. The only metric they had of business success was the bottom line and the annual profits. They failed to recognize the deep inroads the Japanese were making into their markets.

References

[1] http://www.businessmirror.com.ph/the-vietnam-body-count-syndrome/

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s