Monday, September 05, 2011

Type II Error


Even though I did my post-graduate studies in Statistics, but by profession I am not a Statistician. However, data/fact driven analysis and testing of hypothesis always interest me. This article claims that the inherent problem of a human developed system (economic, biological or whatever else it might be) and the reason it dies down over a period of time is that the Type II error rises.
Let me just quote from Wikipedia about the standard definition of Type I & Type II errors:

A type I error, also known as a false positive, occurs when a statistical test rejects a true null hypothesis (H0). For example, if a null hypothesis states a patient is healthy, and the patient is indeed healthy, but the test rejects this hypothesis, falsely suggesting that the patient is sick. The rate of the type I error is denoted by the Greek letter alpha (α) and usually equals the significance level (orsize) of a test.

A type II error, also known as a false negative, occurs when the test fails to reject a false null hypothesis. For example, if a null hypothesis states a patient is healthy, and the patient is in fact sick, but the test fails to reject the hypothesis, falsely suggesting that the patient is healthy. The rate of the type II error is denoted by the Greek letter beta (β) and related to the power of a test (which equals 1-β).

Now, let's take an example.

We all know that when it is red light at a traffic signal you are supposed to stop and go on a green. No errors or mistakes committed. Now, the two errors that can happen are, 1) not stopping at a red light & 2) stopping at a green light.
We all understand that of the above two, it makes sense to reduce occurrences of 1, as it might cause accidents.
When someone jumps a red light, in some places there are sophisticated mechanisms to punish that action by automated means of picture taking etc. or police giving tickets. This is an example of minimizing type I error.
Now, the other error, which is not talked about, is when someone stops at a green light. All of us had that annoying moment when you curse the person in the car in front of you for not moving fast enough at a green light. So, while it does not cause an accident it does cause annoyance. Hence, there is a need for reducing type II error. But, how do you do it?

Let's take another example. Allergy, which is so common in developed world can also be looked at as a type II error or a false negative as well. Body's allergens think it is under attack and tries to resist it causing allergies. It is quite well known that allergy is pre-dominantly a developed country problem. Poorest parts of the world do not face allergy problem, not that much.

Here is another example that my school teacher used to tell us to help explain the concept. He used to say that type I error is like punishing an innocent person and type II error is letting go of a criminal. We can only imagine the horror of a society filled with criminals.

The next example is one of my favorites and very relevant in the United States. You want to have a gun to protect yourself. Not able to protect yourself by not having a gun is type I error. But, misuse and abuse of having a gun in terms of mass shootings is type II error, which is rampant throughout the world today.

The following examples are in the context of computers/internet.

For your email box legit message delivered as spam is type I but a spam message delivered as legit in type II error. We all have experienced that annoying feeling of receiving a spam in inbox!

In the context of hacking, when an authorized user is treated as a hacker is type I error but hackers treated as authorized users is type II, which we definitely don't want.

The next example is related to air travel security screening at an airport. An Innocent traveler considered as terrorist is Type I error and a terrorist considered as innocent traveler is Type II error. We definitely again don't want that to happen.

In all of the examples above, the point is that not able to reduce type II error without comprising type I error is causing havoc in the systems where it happens.
I will revisit this post later to make any corrections and add more arguments etc.


No comments: