Bias, Part Two

Last time we met, there had not been a catastrophic earthquake in Japan like the one that struck on Friday. Even at such a distance, the events there managed to take me away from writing the next "lesson" in the night school. But here I am, and we're going to talk some more about bias. This time we'll look at measurement bias, and how that can have an effect on your findings. So let's look at why it's incredibly important to calibrate your thermometers and mind your decimal places after the jump...

I hope someone calibrated the bungee cord...

INSTRUMENT BIAS
The first kind of measurement bias we'll look at is instrument bias. This kind of bias happens when the instrument you are using to measure outcomes in your research is not up to the challenge. Now, some of you may think that "instrument" refers to something like a thermometer or a caliper only. It doesn't. It also relates to the survey you are using to ask questions of your participants. I touched on surveys a little bit before, and I'll really get into them in a future "lesson" when we cover the three major types of studies (cohort, case-control, point-prevalence surveys).

EXAMPLES OF INSTRUMENT BIAS
Let's say that you are trying to determine if vaccines cause severe fevers after being administered. How would you determine that? One of the ways would be to look at medical records of children who visited the emergency departments in your town and look at children with severe fevers and children without severe fevers. You would then look and see if they received a vaccine shortly before the vaccine. (This is an example of a case-control study.) But what would happen if the thermometers at the different locations were not the same? What if one place takes oral temperatures and the other does tympanic (ear) temperatures only? Can you see how those factors would change what is a "severe" fever and what is not? Of course you can.

Check them for a fever 'cause they're "hot".

Another example is a survey that is not well-designed. Let's say you're trying to figure out how many people in your town have cancer and whether or not that cancer happened from, oh, I don't know... Groundwater contamination with perchlorates from a nearby army facility. If you send out a survey with some questions aimed at finding cases, you better be very careful on how you word the questions and how you analyze the results.

Asking "do you know anyone with cancer?" will likely yield A TON of "yes" results because cancer is not just one disease but a whole spectrum of them. As I'm writing this, I can tell count twelve people in my life who are currently fighting or in remission of cancer. In the analysis, if you say that 90% of respondents answered "YES" and that is a reason to say something is going on... I'll slap you. I will.

He will too.

CONTROLLING FOR INSTRUMENT BIAS
A simple way to control for this kind of bias is to just make sure that all instruments used are standardized (the same) among sites and within participants. In the case of surveys, make sure that the questions are not leading, and that you are being as precise as possible. Instead of asking "Do you know someone with cancer?", it would be better to ask "Do you know someone with (specific cancer name) who lives within 2 miles from (facility in question)?" See? More precise.

RECALL/ATTENTION/EXPECTATION BIASES
I like to clump together these three because they have a lot to do with how humans operate. We evolved to understand patterns and be able to predict certain outcomes based on our experience. When we were running around in the wild, we learned that a sound of hoof beats meant horses, not zebras. We learned that clouds would mean rain, so we sought shelter. What we didn't really get right is probability. That is, we only remember the outcomes of those situations that were of significance to us. It might not have rained every time when there were clouds, but we only remember the times we got caught out in the rain. And we remember that.

Who could forget that rainstorm?

Recall bias is the traditional bias that we learn about in school. It's clear-cut when it comes to a lot of post-exposure studies. Suppose you're asking a group of women if they were exposed to a chemical and how much of it they were exposed to. Now, suppose you ask a group of women with healthy children and another group of women with children with disabilities. Take a wild guess which group reports a higher rate of exposure.

Does that mean that one group is lying? Not at all. It's very human to think that you were not exposed at all if your child is okay. Likewise, if your child has a disability since birth, it's human to think that you MUST have been exposed to tons of the stuff during pregnancy. It's been my experience that charlatans use this to trick you as well. They will ask if you have headaches every now and then. You answer in the affirmative, because we all get headaches. The charlatan will then ask you if you drink water. Who doesn't? When you answer that you do, the charlatan will then use your recall bias and ask you, "Do you remember drinking water before your headaches?" You'll say that you do. The charlatan will then pull out an expensive water purification system... And the rest is history.

So is your bank account.

Sensitivity bias goes along the same lines, but it is present in prospective studies, those that don't rely on your memory for measurements. Imagine that you are studying a pill for headaches. Now, imagine that you have a group of people with chronic migraines and a group of people who have never had a migraine. When told all the wonderful things that the pill may (or may not) do for them, the people with the migraines are more likely to record each and every instance of a migraine they have during the study.

On the contrary, those without migraines are not likely to record each and every instance of a non-migraine (when they're okay). What is at work here is that the people with migraines are paying more attention to the study (the pill, their symptoms, etc.) because they want relief.

No, they DEMAND relief!

Finally, expectation bias has more to do with the researcher than with the participants (or subjects). Without blinding the researchers on which group is