Sorting bias and Mother's Day

It is common to estimate statistical parameters from a large body of data using smaller samples. For instance, polls use a small sample of the public to represent how a larger group feels about a current topic.

If, on average, your estimate does not provide the true value you are trying to obtain, then your estimate is said to have bias. Bias can occur for many reasons. It is possible to introduce bias simply in the way you arrange the data (e.g. from biggest to smallest). This is called sorting bias, and it is common in everyday life.

For instance, your son wants to give his mother (your wife) a gift for Mother's Day. How can you tell which gift he wants to give? You can narrow down the gifts to 2, and hold both potential gifts up to your 9 month old, believing that he'll reach for the one he wants to give. However, if he prefers to reach with a specific hand, the order in which the gifts are displayed will change (or bias) the result. This could be viewed as a form of sorting bias.

As with this example, bias can usually be corrected - giving the baby the same choice in all possible arrangements can remove bias. If he chooses the same gift each time he has no reaching preference. Testing each permutation several times can overcome even mild forms of reaching preference. With severe cases, I worry you are on your own with the whole gift giving process.

Of course, there are large problems with this model - it assumes your baby understands gift giving and thinks his mother would prefer the gift he is picking. This is generally not believed to be true; however, other models require much more experimental data, are too complicated to use on such a simple framework, and are just as difficult to verify as valid.