Twitter competition reveals bias in image-cropping algorithm

DUBAI: Twitter’s image-cropping algorithm prefers faces that are slimmer, younger, and have lighter skin, according to a researcher who took part in a bug bounty competition organized by the social networking company.

The program, started on July 30, invited researchers to hunt for discrepancies as part of Twitter’s first algorithmic bias bounty challenge held at the Defcon convention.

The project was led by Rumman Chowdhury, Twitter’s director of machine learning, ethics, transparency, and accountability for the Middle East, Turkey, and Africa (META), and Jutta Williams, the firm’s product manager for the same region.

In a blog, they said: “Finding bias in machine learning models is difficult, and sometimes, companies find out about unintended ethical harms once they’ve already reached the public.

“For this challenge, we are re-sharing our saliency model and the code used to generate a crop of an image given a predicted maximally salient point and asking participants to build their own assessment.”

The contest’s first prize of $3,500 went to Bogdan Kulynych, a Ph.D. student at the Swiss Federal Institute of Technology in Lausanne. His submission showed how algorithmic models amplify real-world biases and societal expectations of beauty.

Kulynych’s approach consisted of artificially generating faces with differing features and then running them through the algorithm. He found that the algorithm focused on younger, slimmer, and lighter faces over older, wider, or darker faces.

After winning the competition, he highlighted on Twitter the “fast-paced” nature of the contest in comparison to academic publishing. Although he admitted that his submission “came with plenty of limitations that future analyses using the methodology should account for,” he also said it was a “good thing” because even if some submissions only hinted “at the possibility of the harm without rigorous proofs,” the approach would be able to detect such harms early on.

“We should not forget that algorithmic bias is only a part of a bigger picture. Addressing bias in general and in competitions like this should not end the conversation about the tech being harmful in other ways, or by design, or by fact of existing,” Kulynych added.

It is not the first time the biases of the image-cropping algorithm have come to light with several users pointing out the issue last year. At the time, Twitter said in a statement that its team had tested the algorithm prior to launching it but “did not find evidence of racial or gender bias.”

The company pointed out that it would, however, continue its analysis and open source it for others to “review and replicate.”

In a blog post in May, Chowdhury announced the results of the analysis that showed the algorithm favored women to men by 8 percent, white to black individuals by 4 percent, white to black women by 7 percent, and white to black men by 2 percent.

Based on the results, Twitter began testing and rolling out full images in the feed as well as a true preview before posting.

She said: “We’re working on further improvements to media on Twitter that builds on this initial effort, and we hope to roll it out to everyone soon.”

The competition was another step in identifying flaws in the algorithm from an outside perspective.

In their blog, Chowdhury and Williams said: “We want to take this work a step further by inviting and incentivizing the community to help identify potential harms of this algorithm beyond what we identified ourselves.”