Amazon’s facial recognition software wrongly IDs NC lawmaker as a criminal

U.S. Rep. G.K. Butterfield isn’t a crook, but a facial-recognition system Amazon is marketing to police and other groups seems to think he is.

Butterfield, the 71-year-old Wilson Democrat who represents North Carolina’s 1st Congressional District, was one of 28 U.S. senators and House members the software incorrectly matched with criminal mugshots in a test run by the American Civil Liberties Union of Northern California.

The false matches in the set “were disproportionately of people of color” and show why there should be “a moratorium on law enforcement use of face surveillance,” the group said in a statement on its website.

On Friday, Butterfield said he’s “troubled by the inaccurate outcomes associated with this technology, as there are clear blind spots that will have unintended consequences specifically for people of color.”

“While this technology could have far-reaching economic potential, I encourage Amazon to better train its users on best practices for using this technology, be open and upfront about its limitations, and hire more employees of color who can properly assist with addressing the defects of this technology,” he said.

Butterfield is a member of the Congressional Black Caucus, whose members on May 24 wrote Amazon CEO Jeff Bezos to say they were “troubled by the profound unintended consequences this form of artificial intelligence could have for African-Americans, undocumented immigrants and protestors.”

The caucus letter, issued over the signature of chairman, U.S. Rep. Cedric Richmond, D-Louisiana, sought “a substantive dialogue” with Amazon officials and asked Bezos to see to it Amazon hires “more lawyers, engineers and data scientists of color to assist in properly calibrating this technology to account for racial bias that can lead to inaccuracies with devastating outcomes.”

Facial-recognition software is a form of machine learning that in theory can allow users to put names with faces or match old photos to new ones. Amazon calls its version of the technology “Rekognition” and offers it through its cloud-computing service, Amazon Web Services.

It claims the system can provide “highly accurate facial analysis and facial recognition on images and video” users supply.

The ACLU of Northern California says it put that to the test by creating a database of 25,000 publicly available arrest photos, and comparing it to the public photos of the country’s 535 senators and House members. It used the default settings and paid Amazon $12.33 to run the comparison.

The ensuing false positives included three senators and 25 House members. The group was bipartisan, multiracial and gender-inclusive.

An Amazon Web Services spokeswoman, Nina Lindsey, told The New York Times the Rekognition service’s default reports a match if it has 80 percent confidence that two images are alike. Amazon recommends that police departments use a higher threshold, 95 percent, in their work.

The company’s general manager for AI, Matt Wood, in a June 1 blog post said there has been “no reported law-enforcement abuse” of the system.

“We believe it is the wrong approach to impose a ban on promising new technologies because they might be used by bad actors for nefarious purposes in the future,” he said. “The world would be a very different place if we had restricted people from buying computers because it was possible to use that computer to do harm.”

But at least three of Butterfield’s falsely-identified colleagues weren’t amused. Led by U.S. Sen. Edward Markey, D-Massachusetts, they wrote Bezos on Thursday, according to The New York Times, to say the incident raises “serious questions regarding whether Amazon should be selling its technology to law enforcement at this time.”

Their letter asked Amazon to supply Congress more information about the system and its uses by police, starting with the results of “any internal accuracy or bias assessments that Amazon has performed on Rekognition.”

Butterfield has represented the 1st District since 2004, and before that was a Superior Court judge and a N.C. Supreme Court justice. He is a two-time N.C. Central University graduate.

The 1st District now covers most of Durham County and much of the northeastern part of North Carolina.

This story was originally published July 26, 2018 at 7:30 PM.