(Bloomberg View) -- For many people, Donald Trump’s surprise election victory was a jolt to very idea that humans are rational creatures. It tore away the comfort of believing that science has rendered our world predictable. The upset led two New York Times reporters to question whether data science could be trusted in medicine and business. A Guardian columnist declared that big data works for physics but breaks down in the realm of human behavior.

But the unexpected result wasn’t a failure of science. Yes, there were multiple, confident forecasts of win for Clinton, but those emerged from a process doesn’t qualify as science. And while social scientists weren’t equipped to see a Trump win coming, they have started to test theories of voting behavior that could shed light on why it happened.

What’s missing from political forecasting is openness, peer review, and the ability for members of scientific communities to build on one another’s work, according to Simon DeDeo, an astrophysicist who has moved into interdisciplinary work, including social science, at the Santa Fe Institute and Carnegie Mellon University. Secrecy veiled the processes forecasters used to turn polling data into those much-watched probabilities. Many people didn’t ask how the models worked -- they just wanted answers.

Not that these methods are pseudoscience; in fact, they employ some critical tools of science. The most prominent among those is Bayesian statistics, a way of calculating the probability that something is true or will come true.

Bayesian analysis is a core principle laid out in political forecaster Nate Silver’s book “The Signal and the Noise .” Though developed in the 1700s, Bayesian statistics had a resurgence in the science of early 21st century. It revolutionized astrophysics, said DeDeo. Among other things, Bayesian methods allowed scientists to go from wildly different estimates of the age of the universe to a consensus at 13.799±0.021 billion years. They can also predict how the universe will end (not well, with or without Trump).

Why don’t Bayesian statistics work the same sort of consistent magic for political forecasts? In science, what matters isn’t the forecast but the nature of the models. Scientists are after explicit rules, patterns and insights that explain how the world works. Those give other scientists something to build on -- allowing science to self-correct in a way that other intellectual ventures can’t.

A scientific model could be very simple, said DeDeo. Maybe your model posits certain kinds of people surveyed in polls don’t admit their choices to pollsters. Or that certain kinds of people will say they plan to vote but won’t follow through. As long as the ideas are testable, and the results open to peer criticism, the power of science can move things ahead.

The problem with opaque models, he said, is you never know why you might be right or wrong. If a forecaster gives Trump a 15 percent chance of winning and he wins, does that mean the model is bad? Or could the model be great and used to predict many future elections -- it was just that the more improbable outcome happened this time around? DeDeo explained this problem in a not-yet-published paper, “Wrong for the Wrong Reasons: Lessons on November 9th for the Science of Human Behavior,” which he said he wrote on that day.

Some people predicted a Trump win using models that ignored polling data, he said, instead using a series of historical factors -- whether there was a scandal during the previous administration, for example, and whether there was a perceived foreign policy success. This isn’t a scientific model yet, but it could be explored scientifically.

Now that it’s over, there’s still a chance for science to explain why so many people voted for Trump. There are all kinds of guesses and judgments being thrown around about Trump voters -- that they’re racist or sexist, or responding to the call of tribalism. Those aren’t the least bit scientific, but they could be turned into testable hypotheses.

On the day after the election, University of Pennsylvania psychologists Robert Kurzban and Jason Weeden presented a case that people were voting -- or not voting -- out of simple self-interest. They summed up their idea in a Washington Post column titled “No, Voters Are Not Irrational.” For their idea to work, they factor in status as well as money to the self-interest equation.

People who live very monogamous lifestyles, for example, may benefit from living in societies with strict rules about contraception and abortion. This may explain why many evangelicals were willing to forgive Trump’s personal sins in order to get a conservative Supreme Court justice. Or, the psychologists suggest, white males without higher education may benefit from Trump’s promise to limit sources of competition for scarce jobs.

Kurzban says this doesn’t account for the fact that some of the candidates’ promises are more believable than others -- and that people vary in their willingness to believe them.

And this is not exactly the rationality that people value as a guide to truth. It’s the kind of animal rationality that motivates a chimpanzee to take a shot at advancement by clawing a rival to shreds. If such competitive instincts motivate our behavior, it’s worth understanding.

But if that were the only form of rationality we followed, how would we ever have developed science to the point that we knew we were closely related to chimps? Or that we’re primates? That’s why there is a whole field of study devoted to the calculus of cooperation, attempting to explain why people sacrifice for others.

Social scientists didn’t try to forecast the election, and rightly so. They weren’t up to the task. But someday, they might be.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access