Ready or not, the presidential primary season is quickly on its way. Even at this early stage, there's no shortage of declared candidates – at last count 15 Republicans and five Democrats. Should be quite an exciting 16 months.

Much more blue than red, I nonetheless track both sides closely, as much to examine the “evidence” they offer to support their policies/accomplishments than for political views per se. Sadly, both blue and red compete for the dubious “pinnochios” awarded by the Washington Post's Fact Checker for political truth stretching.

In Chicago, we hear quite a bit about woebegone Illinois. It seems all our Great Lakes neighbors have much fun at our expense, and for good reason  – what with Illinois' horrendous budget deficits and pension funding problems exacerbating middling job and growth performance. We do, however, enjoy monitoring political developments of others in the region, especially when they offer us the comfort of schadenfreude.

Wisconsin governor Scott Walker, the just-announced Republican candidate, has been particularly newsworthy lately, with lots of Chicago coverage of both his successes and travails in Badgerland. Having followed Walker”s ascendance closely over the past five years, I must admit to being as suspicious as those who propose Walker's time in Madison is being spent doing little more than auditioning for the presidential stage he covets.

The governor proffers major accomplishments in Wisconsin as springboards to the Presidency. He somehow claims his blueprint used to defeat government labor unions will generalize to foreign policy successes. “If I can take on 100,000 protesters, I can do the same across the world.” 

Walker also highlights substantial economic gains for Wisconsin  under his tutelage. “Today, the unemployment rate has dropped to 4.6 percent, more than 30,000 net new businesses have been created, and in 2014, Wisconsin ranked ahead of the nation in growth in per capita personal income. Wisconsin has created nearly 150,000 private sector jobs since December 2010.”

My problem with such bravado is that both the success measures and the methodological designs for “proving” the merits of such policy are generally unscientific and self-selected by the beneficiaries. Indeed, as we emerged from a deep recession in 2010, when Walker was elected, a fair amount of economic lift was the inevitable consequence of a rejuvenating business cycle.

Aggrandized politicians are guilty of the post hoc ergo propter hoc fallacy:  “simply because one thing happens after another, the first event was a cause of the second event.” I suspect Wisconsin would have experienced a noticeable drop in unemployment during Walker's time in office had Homer Simpson been governor.

About a year ago, I posted a blog that addressed this absence of methodological rigor in political policy evaluation. What we often see is policy-based evidence-making masquerading as evidence-base policy-making. For the latter, rigorous designs asking not only “what” and “how big”, but also “compared to” must be executed. Yes, Wisconsin under governor Walker has witnessed an improvement in economic indicators, but how does that uptick stack up to the performance of similar states?

The gold standard design for determining cause and effect of an intervention, the randomized experiment, is often not accessible in public policy. But rigorous, natural, quasi-experimental methods can generally be applied.

Consider the  “interrupted time series with non-equivalent control group design”, long deployed in the academic world and now a staple in data science. Measurements from the “experimental” group are compared to a natural control over time, with special emphasis on the occurrence of a change or interruption.

To evaluate governor Walker's claims, the Wisconsin economy might be compared to the control of neighboring Minnesota, with key historical economic indicators measured preceding and following the election of Walker. If the governor's claims are valid, Wisconsin should look good by the numbers in comparison to Minnesota.

Alas, the economic “experiment” with Minnesota as control is unflattering to both Wisconsin and the governor. On just about all measures of state economic performance contrasting Wisconsin and Minnesota published by the Federal Reserve Bank of Philadelphia and the Bloomberg Economic Evaluation of States,  the Gopher state wins hands down. And the differences have become more pronounced over the last five years – since Walker assumed office.

Neither source minces words. Bloomberg opines the data show "that measured by relative economic outcomes, Walker's tenure falls somewhere between lackluster and a failure." Representing the FR economic activity index, MPR News adduces that "Minnesota and Wisconsin ran neck-and-neck for years, then Minnesota started to pull away in the late 1990s.....That trend has accelerated since the Great Recession. Minnesota is leaving Wisconsin behind."

A warning for 2016:  Be wary of policy-based evidence-making from the candidates. Leave the debates to data scientists.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access