Microsoft Researcher Details Real-World Dangers of Algorithm Bias

["Dave Farber" <farber () gmail com>]

> https://gizmodo.com/microsoft-researcher-details-real-world-dangers-of-algo-1821129334 
> <https://gizmodo.com/microsoft-researcher-details-real-world-dangers-of-algo-1821129334>
>
> Microsoft Researcher Details Real-World Dangers of Algorithm Bias
>
> Sidney Fussell <//kinja.com/sidneyfussell>Yesterday 5:00pm 
> <https://gizmodo.com/microsoft-researcher-details-real-world-dangers-of-algo-1821129334>
>
> Screengrab: Kate Crawford’s “The Trouble With Bias” at NIPS 2
> However quickly artificial intelligence evolves, however steadfastly it becomes embedded in our lives—in health 
> <http://fortune.com/2017/10/30/ai-early-cancer-detection/>, law enforcement 
> <https://www.washingtonpost.com/local/public-safety/police-are-using-software-to-predict-crime-is-it-a-holy-grail-or-biased-against-minorities/2016/11/17/525a6649-0472-440a-aae1-b283aa8e5de8_story.html?utm_term=.37a7d249ff8a>,
>  sex <https://gizmodo.com/the-future-of-online-dating-is-unsexy-and-brutally-effe-1819781116>, etc.—it can’t outpace 
> the biases of its creators, humans. Microsoft Researcher Kate Crawford delivered an incredible keynote speech, titled 
> “The Trouble with Bias,” at Spain’s Neural Information Processing System Conference on Tuesday. In Crawford’s 
> keynote, she presented a fascinating breakdown of different types of harms done by algorithmic biases.
>
>
> As she explained, the word “bias” has a mathematically specific definition in machine learning, usually referring to 
> errors in estimation or over/under representing populations when sampling. Less discussed is bias in terms of the 
> disparate impact machine learning might have on different populations. There’s a real danger to ignoring the latter 
> type of bias. Crawford details two types of harm: allocative harm and representational harm.
>
> “An allocative harm is when a system allocates or withholds a certain opportunity or resource,” she began. It’s when 
> AI is used to make a certain decision, let’s say mortgage applications, but unfairly or erroneously denies them to a 
> certain group. She offered the hypothetical example of a bank’s AI continually denying mortgage applications to 
> women. She then offered a startling real world example: a risk assessment AI routinely found that black criminals 
> were a higher risk  <https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing>than 
> white criminals. (Black criminals were referred to pre-trial detention more often because of this decision.)
>
> Representation harms “occur when systems reinforce the subordination of some groups along the lines of identity,” she 
> said—essentially, when technology reinforces stereotypes or diminishes specific groups. “This sort of harm can take 
> place regardless of whether resources are being withheld.” Examples include Google Photos labeling black people as 
> “gorillas,” 
> <https://www.usatoday.com/story/tech/2015/07/01/google-apologizes-after-photos-identify-black-people-as-gorillas/29567465/>
>  (a harmful stereotype that’s been historically used to say black people literally aren’t human 
> <https://theconversation.com/comparing-black-people-to-monkeys-has-a-long-dark-simian-history-55102>) or AI that 
> assumes East Asians are blinking when they smile 
> <http://content.time.com/time/business/article/0,8599,1954643,00.html>.
>
> Crawford tied together the complex relationship between the two harms by citing a 2013 report from LaTanya Sweeney 
> <https://www.technologyreview.com/s/510646/racism-is-poisoning-online-ad-delivery-says-harvard-professor/>. Sweeney 
> famously noted the algorithmic pattern in search results whereby googling a “black-sounding” name surfaces ads for 
> criminal background checks. In her paper, Sweeney argued that this representational harm of associating blackness 
> with criminality can have an allocative consequence: employers, when searching applicants’ names, may discriminate 
> against black employees because search results are tied to criminals.
>
> “The perpetuation of stereotypes of black criminality is problematic even if it is outside of a hiring context,” 
> Crawford explained. “It’s producing a harm of how black people are represented and understood socially. So instead of 
> just thinking about machine learning contributing to decision making in, say, hiring or criminal justice, we also 
> need to think about the role of machine learning in harmful representations of identity.”
>
> Search engine results and online ads both represent the world around us and influence it. Online representation 
> doesn’t stay online. It can have real economic consequences, as Sweeney argued. It also didn’t originate online—these 
> stereotypes of criminality/inhumanity are centuries old.
>
> As Crawford’s speech continued, she went on to detail various types of representational harm, their connections to 
> allocation harms and, most interestingly, the ways to diminish their impact. As is often suggested, it seems like a 
> quick fix to either break problematic word-associations or remove problematic data, what’s often called “scrubbing to 
> neutral.” When Google image search was shown to have a pattern of gender bias in 2015, showing almost entirely men 
> <https://www.eurekalert.org/pub_releases/2015-04/uow-wac040915.php> when users searched for terms like “CEO” or 
> “executive,” they eventually reworked the search algorithm so it’s more balanced. But this technique has its own 
> ethical concerns.
>
> “Who gets to decide which terms should be removed and why those ones in particular?” Crawford asked. “And an even 
> bigger question is whose idea of neutrality is at work? Do we assume neutral is what we have in the world today? If 
> so, how do we account for years of discrimination against particular subpopulations?”
>
> Crawford opts for interdisciplinary approaches to issues of bias and neutrality, using the logics and reasoning of 
> ethics, anthropology, gender studies, sociology, etc, and rethinking the idea there there’s any one, easily 
> quantifiable answer.
>
> “I think this is precisely the moment where computer science is having to ask much bigger questions because it’s 
> being asked to do much bigger things.”



-------------------------------------------
Archives: https://www.listbox.com/member/archive/247/=now
RSS Feed: https://www.listbox.com/member/archive/rss/247/18849915-ae8fa580
Modify Your Subscription: https://www.listbox.com/member/?member_id=18849915&id_secret=18849915-aa268125
Unsubscribe Now: 
https://www.listbox.com/unsubscribe/?member_id=18849915&id_secret=18849915-32545cb4&post_id=20171209114935:EE4D625E-DD00-11E7-8D5E-CFC5472FC3D0
Powered by Listbox: http://www.listbox.com