The need for analytic and algorithm governance is growing
I wrote a blog in 2017 titled, The Inflated Price of Perfect Information. In that blog I wrote the following:
Two articles in the last week show the fallacy of the idea of perfect information. In the Economist this week there is “How Price-bots can conspire against consumers – and how trustbusters might thwart them“. This article reports on an illicit cooperation between price bots setting gas prices in the US. In today’s Wall Street Journal there is an article titled, “To Set Prices, Stores turn to Algorithms” that reports on a similar story in Germany. The point of the message is that when algorithms are charged with maximizing certain objectives, they might accidentally “collude” (more precisely, behave as if they were colluding since they cannot) and the result has been, so the observations states, a rise in prices. How can this be?
Why do I note this today? In yesterday’s US print edition of the Wall Street Journal there was a fine insert with the title and focus: Artificial Intelligence. It is a quick snapshot on the state of the market of AI. One startling item concerns an innovation being worked on that seems to suggest that those of us that are ill with heart disease, but might not yet know it, might be helped simply by talking to a specific app trained with AI to watch for certain heart disease markets. Yes, markers that can be found in ones’ voice!
However there is another article titled, Beware Algorithms That Could Collude to Unfairly Raise Prices. The article reports on an experiment where competing algorithms were given price maximization goals and they were left to compete. It turns out that the result were artificially elevated prices. The concept is that the AI’s colluded – though of course they cannot actually collude. They just react to the change in price set by the other algorithm. In other words, its a kind of phantom collusion. The WSJ article uses the term tacit collusion, as reported the researchers who led the experiment.
The definition of collusion seems to be of the form: secret or illegal cooperation or conspiracy, especially in order to cheat or deceive others. There is nothing secret going on between algorithms; they don’t sneak behind the bike shed to hatch a price-setting plan. But the point is clear – and was quite popular a couple of years ago. Many prices (and decisions to buy or sell) are automated with rules, and now more with machine learning and AI capabilities, ranging from stocks and bonds, to oil and gas, and other commodities. Retailers have been using neural networks to optimize prices of baskets of good for years, in order to exploit shopping habits.
The bottom line here is that designing and building algorithms can only go so far. Organizations need to invest in soft skills related to governing the behavior of algorithms and AI-based solutions. This topic is gaining in popularity but it lags the popularity of acquiring some AI capability in general. As such, the outcomes we see will always likely thrill or concern us more than they should; until firms get a good handle on how to implement AI and algorithm governance: data and analytics governance.
(This post originally appeared on Andrew White's Gartner blog, which can be viewed here).