Mental exercises for a better brain

There’s this discussion going on over at Respectful Insolence between an anti-vaccine activist and an epidemiologist, like me. The anti-vaccine activist — whom I thought was banned from there (oops) — is known to be quite “dense” when it comes to epidemiology and biostatistics. I don’t blame him, much. His highest degree in science is in Fire Science. I don’t know where this guy when to school, but most programs I’ve found, like this one, don’t have biostatistics or statistical reasoning in their curricula. This would explain the activist’s misunderstanding of a case-control study. Like the PhD in Biochemistry being discussed by Orac in that post, the activist thinks that matching cases and controls in a study somehow disallows for the examination of their vaccine status and its relationship to autism. They think that cases (autistic children) should have a different vaccine status than controls (neurotypical children), and then we can see if they have a difference in vaccine exposures.

Can you see the logical fallacy in that?

As Ren explained over and over and over and over and over to the activist, you cannot and should not match cases and controls on the variable you’re interested in analyzing. He even used an excellent mental exercise:

“Suppose that you had an outbreak of intestinal disease in people who went to a wedding. Your cases are people with diarrhea and your controls are people without. If we were to do the case-control investigation the way you want it, we’d get people who went to another wedding and were not sick to compare to the sick people from our wedding. (Well, not our wedding…) If we did that, the only thing we could definitely conclude is that the sick people have greater odds of being at our wedding than the other one. Period.

Of course, we want to find out what food made people sick, so we need to be smarter about getting our controls. So we get our controls from people at the wedding, but we stupidly pick controls who are children versus the cases who are mostly adults. I don’t know about your wedding, which I’m sure was fabulous, but kids usually have a different menu at weddings. We do the comparison and we can only say that people who fell ill had greater odds of being adults at the wedding.

You can see where I’m going with this. See, we need to match the controls to the cases in all respects except the food. We don’t ask about the food in choosing controls. We can’t say, “Hey, if you ate the roast beef, you can’t participate,” because we wouldn’t be able to say anything about the odds of eating roast beef as it relates to the disease. As you can see, both our cases and controls have equal odds of their age, being at the wedding, etc… Except the food. That is the unknown.

We ask them what they ate and, almost magically, we see that people who were sick had greater odds of having eaten, say, the pasta dish.

Also, although I covered this before in the Epi Night School, note that we’re not saying that eating the pasta dish increased the risk of diarrhea in the group. We can’t. With case control studies, we can only say that those with diarrhea had greater odds of eating the pasta dish. It may sound like semantics, but it’s important. It doesn’t prove causation. For that, we use the scientific plausibility of pasta causing diarrhea… So we test the pasta for pathogens, bad storage conditions, etc. Ninety-five times out of 100, we’ll be right… And that’s an acceptable p-value.”

I couldn’t have written that better myself.

But let’s do a mental exercise where we please the anti-vaccine activist and his followers and do a “vax v. unvax” study, while trying not to violate most if not all rules of ethics. Let’s take children from City A, a city where there are no vaccinated children, and City B, a city where vaccination rates are reasonably high at 90%. Both cities have a population of 100,000, are in Missouri, and everything else is equal in that we’re not worrying about race, ethnicity, socioeconomics, etc. Also, all the children in this study are under the age of 5. I’ll be using the US Census Bureau‘s data on Missouri for this example. This is what we have so far:

 

So we randomly pick 650 unvaccinated children from City A and, say, 1,300 vaccinated from City B. That’s a reasonable 1:2 ratio of cases to controls. And let’s say that the prevalence of autism is 1:50, as the latest data says. We then go back and ask the parents of the children in both groups if their children have autism, and this is where we run into trouble. We run into trouble because parents of an autistic child are more likely to stop or delay vaccination once their child is diagnosed because of fear that the vaccines caused the autism to begin with. Also, parents who dislike vaccines are less likely to seek medical care from trained professionals who would diagnose their children with autism. You would also see this effect coming out of City B, where parents who vaccinate are more likely to take their children to healthcare providers who can diagnose autism. These things are biases.

Anyway, if the autism prevalence in both cities is 1 in 50 (2%), the Odds Ratio is 1.0. (Odds Ratio is the odds of being vaccinated if you have autism.) In this example, both groups have equal odds of being vaccinated:

OR = (26*637) divided by (1274*13), or (Cell A * Cell D) divided by (Cell B * Cell C)

It makes sense, right? Both cities have an equal autism prevalence. Ah, but anti-vaccine people will tell you that there should be zero children in City A with autism. I don’t like to deal in absolutes, so let’s just bring that number to one.

 

 

Wow!

That’s right. If we are to believe anti-vaccine people, then autistic children would be 20 times more likely to be vaccinated than unvaccinated (in this exercise). Twenty times! But is that the truth? Can that be true? Can you find a population with absolutely 0% vaccine rate and virtually 0% autism? Even the Amish vaccinate and have autism cases, despite anti-vaccine claims to the contrary. Furthermore, how do you deal with the biases I explained above?

Finally, remember that the autism prevalence in the two cities is the unknown we’re trying to determine, meaning that we’d have to change this graph around since the dependent variable would be switch places with the independent variable. That would give us all sorts of funky numbers because things divided by zero give non-numbers and things multiplied by zero equal zero. Like Ren told the anti-vaccine activist at Orac’s, it’s math, not magic.