My wife and I recently enjoyed a nice evening in the neighborhood with six couples we've known since our college daughters' kindergarten days. The girls are off in many different directions now, and we don't get to see each other as much as we used to, so it was especially fun to catch up and reminisce.

As is the custom for at least a portion of such events, the men gather in the family room to watch a game and discuss sports and politics. The conversations can become pretty “enthusiastic”, as the suburban Chicago neighborhood is a mixture of both red and blue political persuasions. This time, admonished by my wife to behave, I sat on the sidelines and watched the others entertain.

The main debate revolved on an interview with Hilary Clinton in which the unannounced candidate contrasted the economy of husband Bill's 1990's very favorably with the 1980's under Ronald Reagan. In the “discussion” that followed, the “red” group pined for the halcyon Reagan days, while the “blues” lamented the lost Internet boom after Clinton. For the reds, everything touched by Reagan turned to gold. Ditto for the blues and Clinton. Me? Economically, I'll take either period over what we've experienced since.

Watching the debate, I was able to “appreciate” the unconvincing arguments of both sides. Reagan did such and such that resulted in dissipating the sins of Carter and stagflation of the 1970's, it was argued. The 1995-2000 period under Clinton's stewardship was a boom period the likes of which we may never again see again – and vanished soon after Clinton left office, was the counter. Yada, Yada. And, of course, it was each president's decision-making and directives that “caused” the prosperity. Both sides' arguments claimed support for their party's policies based on the salutary resulting outcomes.

As I sat silently waiting for the group to solicit my input, I recalled the wisdom of Duncan Watts in a blog I'd recently posted. Watts seeks “evidence-based skepticism” to deal with contentions surrounding the results of policies, interventions or business strategies:
When someone says, ‘‘Oh, you know, this is how things work,’’ the question should be,‘‘Well, okay, what is the evidence?’’ Right? Not just, ‘‘Oh, that is a plausible assertion. I am going to believe you, because it conforms with what I already think.’’ But, “Okay, that is plausible, but there are five other things that are also plausible. Why is your plausible assertion more correct than any of these other alternative hypotheses?” We just do not really think that way.

When finally I was asked for my perspective, I opined I didn't believe either side. Both based their contentions on the flimsiest post hoc ergo propter hoc (after this therefore because of this) designs – fallacies “based upon the mistaken notion that simply because one thing happens after another, the first event was a cause of the second event.” Without sounder comparison methodologies that include control groups, there is no end of creative – and convenient -- explanations that can be divined to support policy successes and failures of each. I'm not so sure the 80's and 90's wouldn't have been prosperous had I been president.

Relevant Wattsian skepticism was on display in spades in a wonderful recent Upshot article by economist Justin Wolfers. Probably more blue than red, Wolfers nonetheless even-handedly analyzed the impact of the legislative discontinuation of jobless benefits in North Carolina.

Emotional investment in the decision was high. The reds, “who voted against extending unemployment benefits, argue that ending benefits will spur the long-term jobless to look harder for work....” In contrast, many blues “voted to continue jobless benefits for the long-term unemployed, say(ing) that ending benefits will force the unemployed to cut their spending, which may have broader ripple effects that could slow the labor market recovery.” Not surprisingly given the improving economy, both sides claimed victory, yet offered only weak post hoc arguments and convenient friendly statistics in support.

Ever the skeptic, Wolfers demands more evidence – and doesn't find it in the arguments of either camp. Without the luxury of an experiment that randomly assigns the unemployed to either continuation or cancellation of benefits groups, he cleverly finds “natural” control groups to “analyze” the success claims of both the red and blue.

To support their position, the reds adduce that NC employment grew by 1.5 percent in the six months post discontinuation. Wolfers notes that “while this growth rate is healthy, what matters here is whether it is better or worse than it would have been without such a policy shift.” Enter neighboring state and natural control group South Carolina, “an otherwise similar state that made no such change.... Over the same period, nonfarm payrolls in South Carolina grew by 1.6 percent.” Kind of takes some of the wind out of the red sails.

Six months after NC discontinued its program, the federal government dropped its long-term unemployment benefits as well. Now, North Carolina, which was already without benefits, became the control group for South Carolina. The blues would argue that SC, which just lost its federal benefits, would suffer in comparison to NC. The results? “over the six months since federal benefits ended, nonfarm payrolls grew by 0.6 percent in South Carolina and 0.4 percent in North Carolina. Again, the trends in the two states remain strikingly similar.”

Wolfers' take? “How, then, have advocates come to different conclusions? They have relied on evidence that isn’t really evidence, often to support conclusions that fit their pre-existing views. It can make for claims that sound persuasive on the surface but do not stand up to scrutiny.”

Perhaps Brookings Fellow Richard Reeves has it right when he suggests the world would be a better place if there were a little more evidence-based policy-making and a lot less policy-based evidence-making.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access