Amazon schooled on AI facial tech by Turing award winner

Register now

(Bloomberg) Inc.’s facial-recognition software is coming under fire again, this time from a group of artificial intelligence researchers that includes one of the field’s most prominent scientists.

Twenty-six researchers, including Yoshua Bengio, a recent winner of the Turing Award, the industry’s highest honor, called for Amazon to stop selling its Rekognition AI service to police departments in a post on Wednesday. Bengio was joined by Anima Anandkumar, a former principal scientist at Amazon’s cloud division, and staffers from Google, Microsoft Corp., Facebook Inc. and several universities.

The group defended the work of two other AI researchers who found Amazon’s software had much higher error rates when predicting the gender of darker-skinned women in images, compared with lighter-skinned men. Amazon had argued against the results and methodology of that study, authored by the University of Toronto’s Inioluwa Deborah Raji and Joy Buolamwini, a researcher at Massachusetts Institute of Technology.

Amazon did not immediately reply to requests for comment.

Almost everyone involved in facial-recognition technology sees problems, even as the software is being used for everything from tagging friends in social media photos to law enforcement. The dispute over Amazon’s software has been raging for almost a year. The company sold its Rekognition product to a police department in Oregon, sparking concern it could fuel discrimination. The American Civil Liberties Union said a test it ran of the software falsely matched 28 members of Congress with mugshots and that those flagged were disproportionately people of color.

“It is important to have the social and scholarly debates about what is socially and ethically acceptable in the use of these new technologies,” Bengio said in an email. “This case highlights such issues in a very clear way and is a good way to increase public awareness.” Bengio, a professor at the University of Montreal, said he hopes the discussion will led to companies putting in place internal rules as well as the adoption of government regulations so that “the best-behaving companies are not at a disadvantage compared to the others.”

Matt Wood, general manager of AI at Amazon Web Services, and Michael Punke, vice president of global public policy, said the company couldn’t reproduce the results of Raji and Buolamwini’s study. The Amazon executives also said the study did not use Rekognition in the right way and that it used one type of facial-AI software as a proxy for another.

Bengio, the Turing Award winner, and the other 25 AI researchers called Amazon’s response “disappointing” on Wednesday.

“We hope that the company will instead thoroughly examine all of its products and question whether they should currently be used by police,” the researchers wrote. “Mr. Punke writes that the company supports legislation to ensure that its products are not used in a manner that infringes civil liberties. We call on Amazon to stop selling Rekognition to law enforcement as such legislation and safeguards are not in place.”

The researchers also noted that other companies, including IBM and Microsoft, were able to reproduce the original study’s data and use it to improve their facial-recognition systems.

“The fact that company representatives chose to refute the Raji and Buolamwini paper highlights the importance of a rational and open debate, which will hopefully discourage other companies from using similar tactics, and instead encourage them to improve their products appropriately and engage in a constructive dialogue with scientists who work on these issues,” Bengio said.

Bloomberg News