Last week John Desmond presented an interesting article that introduced me to an increasingly and serious topic regarding social prejudices at some artificial intelligence (AI) applications. It ain’t a new or unknown problem, but it has produced increasing awareness from governments, professionals and developers during this last few years.
The problem is that, a tendency has been identified for segregation according to physical appearance and gender during the run of some AI algorithms for generating people management data.
Image EFF Photos (CC BY)
Arvind Narayanan, an assistant professor of computer science at Princeton and data privacy expert co-authored a paper last year where he analysed the interpretation of some words by means of AI. His studies showed that both humans and AI software related male names with words like executive and female names with marriage. The software also perceived European American names (Paul, Ellen) as more pleasant than African American ones (Malik, Shereen).
According to a recent report by ProPublica, a software tool used by some US courts was suspected to be biased against African-Americans. The software predicted future crimes and recidivism of convicted persons and it rated African-Americans with higher risk of committing crimes again as twice the rate of white people, and underestimating future crimes among white people.
(CC BY-SA) Abyssus, from Wikimedia Commons
As we can hear from these problems, there’s also an interest to create a push back against them. Examples are the following:
- Nonprofit organizations such as AI Now Institute and OpenAI are proposing that “if an algorithm providing services for people cannot explain its decision, it should not be used”.
- On 25 May 2018, the General Data Protection Regulation (GDPR) will be enforced across Europe imposing fines to non-compliance organizations. The GDPR sets a series of rights of the data subject under three key principles:– fair and lawful processing;
– purpose limitation;
– data minimization and data retention. - Bias identification and correction startups are beginning to provide businesses with technical support like the Online Risk Consulting & Algorithmic Auditing.
- At this year’s Fairness, Accountability, and Transparency (FAT) conference held in NYC last February, there had been over 900 computer scientists interested in Machine Learning concerns regarding this topic.
In conclusion, it kept me reflecting on how powerful such data generation, evaluation and management can become for our current society. We are living in the so called fourth industrial revolution (Industry 4.0) where we are becoming to rely on technologies such as AI to make our processes efficient and automated. Developers and managers must be taking the necessary actions to filter not only algorithms but the way they’re being developed to avoid any kind of bias that generates social conflicts.
I’ll be happy to hear your thoughts about this topic; I encourage you to read more about it in the links I’ve included in this publication.
I leave you with what Sandra Wachter, a lawyer and researcher in data ethics at London’s Alan Turing Institute and the University of Oxford said: “Algorithms force us to look into a mirror on society as it is”.
Eng. Juan Zamora 