I think Jim Manzi nails it.
Russ: Yeah, I think we have a lot of romance about randomized control trials like this, because they remind us of science.
It’s like, we’ve got the petrie dish over here; it has whatever; and another one over here. And that’s science! So if we have a control group and an experimenal group, we’re going to find the truth. We’re not going to be confused by–and one thing I think listeners can take from this conversation and related conversations is the elusiveness of truth. That it’s much harder, as you point out in your book Uncontrolled, and I think you do it very beautifully, in a world of what you call ‘causal density,’ where there’s lots of different things happening and changing at the same time, and there are a lot of unobservable differences between groups, you should lower your expectations.
Guest: Yeah, that’s right. And I think in a very specific way. I think you really can find truth in a scientific sense. You really can find the right answer, the true answer to these questions about causality. It’s just an experiment answers an extremely narrow question, always. It’s not: Does Medicaid help or hurt? You know, what you answered is: When I randomized people to this lottery in Oregon on these dates, what was the causal effect of being randomized in or out of that lottery? And the reason you’ve heard me hedging all of the time is I’ve learned the hard way, as soon as you step an inch off that platform, not seemingly grandiose, but seemingly direct implications of that, you run into the danger of fooling yourself into thinking your knowledge extends more broadly than it does. That’s when you need lots and lots of experiments. You build this picture of knowledge by a pointilist painting. By lots and lots of lots of experiments that get the answer in very narrow circumstances. But you can add those up to really useful conclusions. I think.