Psychology of bias and its relation to algorithms
Before delving into specific examples of bias within the design of algorithms, we first have to obtain a definitive definition, perspective of the word ‘bias’ and its formulation into each one of us as a species. It is important to first understand the nature for which we interpret bias. This then allows us to analyse algorithms with greater knowledge of their causality as opposed to just quantifying them down to a set of calculations and code, all of which are designed and developed by humans prior to any end user interaction or the experiencing of the repercussions of them.
Cognitive bias (bias) can be described as a “Inclination or prejudice for or against one person or group, especially in a way to be considered unfair” (Oxford dictionary of English, 2005). Suresh and Guttag (2019) in their paper “A Framework for Understanding Unintended Consequences of Machine Learning” state their definition of bias is as one that “refers to an unintended or potentially harmful property of the data”. In this definition “data” refers to the input or output of a machine learning algorithm. Oxfords’ definition provides great understanding to the cultural aspects in which bias affects us as individuals, while Suresh and Guttag allow this cultural definition to be focused on the output effects of an algorithm, still in context to its implications for the end user.
We commonly take bias as an auto-unjust definitive of human social interaction but it is in fact ingrained in all of us by nature. Referring to Darwin’s theory of natural selection (Darwin, 1906) which could also be classified as an organic algorithm: where organisms “multiply”, mutations in this organism “vary” and finally those organisms with the correct mutations live and those without die, what at first can be presented as something with dramatic means can give cause to prosperous ends. These mutations and eradication of flaws lead to a species with apt ability to survive in its surroundings. An instinctual example of bias within humans is the sexual preferences of females. Females with a higher standard of physical attractiveness find masculine males more attractive than females of a lower standard of physical attraction (Little et al., 2001). To put this in a digital perspective, female users of the dating app Tinder receive a match rate of 10.5% whereas males receive a match rate of 0.6%; firstly showing that males try to match at a much higher average than females and secondly that females are much more particular with whom they match with. This is all without the addition of a sorting algorithm based upon user preference, and only upon the user selected variables of age and relative location (Tyson et al., 2016). These examples of bias in nature highlight the differing sexual agendas between males and females, possibly attributed to reproductive interests and the asymmetries that exist in levels of parental investment between the two sexes (Trivers, 1972). Being bias while choosing a mate is therefore implicit to minimising reproductive costs imposed by the other sex due to the previous mentioned asymmetries, and therefore necessary should humans wish to reproduce efficiently (Muller and Wrangham, 2009).
“In sum, there may be many evolutionary reasons for apparent design flaws, and a close examination often provides insight into the evolutionary forces that shaped them and their functions. We may propose that analogous logic may be applied to understanding cognitive biases.”
- (Haselton, Nettle and Andrews, 2005)
Commonly all biases can be interpreted in two different ways: being conscious (controlled or explicit) and unconscious (automatic or implicit). Conscious bias involves direct action by an individual to insight said bias. Unconscious bias on the other hand exists in individuals without purposefully inciting its introduction, but more so introduced though exposure (Brainerd, Stein and Reyna, 1998 see Freud and Rieff, 2008; Jung and Hull, 2014 for further reading). An example of how exposure dictates human perception can be seen in India where mandatory gender representation assignments of village leaders was investigated. Male voters were automatically bias towards hypothetical female leaders even if said leader was outperforming their hypothetical male counterparts. However, they also found that after 10 years of mandatory female representation, woman were more likely to stand and win political positions; dispelling their bias (Beaman et al., 2008). This example demonstrates how biases such as stereotypes sometimes have to be forcefully broken and presented for a system to recognise its discrimination.
Jost and Banaji (1994) provide reasoning to why individuals or groups within a society stereotype others. They highlight the two justifications for stereotypes, ego-justification and group-justification, and later propose a third, system-justification, to address principles not encompassed within the two previous. The authors state, “ego-justification refers to the notion that stereotypes develop in order to protect the position or behaviour of the self” and that “Group-justification views assume that stereotyping emerges in the service of protecting not just the individual ego, but the status or conduct of the social group as a whole”. They then propose that “system-justification”, unlike both previously mentioned justifications of stereotypes, “does not offer an equivalent function that operates in the service of protecting the interests of the self or the group” but are actually acknowledged and allowed to persist in spite of having a negative mental or material impact. Going back to Beaman et al.’s, (2008) example of Indian leaders, where male voters were discriminating against female leaders, this would represent system-justification where even if the hypothetical female leader was performing better to their male counterparts, they still discriminated even though their lives could ultimately be affected positively as a result. It is important to consider stereotype justifications and their implications within the design of algorithmic methods. As we will explore further in this paper, individuals (designers/developers), groups (big data companies) and systems (the tech industry) will aim to justify the inequality promoting actions carried out by their systems.
Suresh and Guttag (2019) propose five comprehensive terminologies that aim to effectively communicate causality to fairness within machine learning and algorithmic design:
Historical Bias
Historical biases arises when stereotypes are reinforced, commonly relating to specific identity groups. These biases can still commonly creep into an algorithm even if the data is perfectly measured and sampled correctly due to pre-existing social biases contained within the data sets such as race segregation in American cities (Chang, Posner and Lee, 2019). Suresh and Guttag use the example of Google image search to propose the morality of historical bias pointing out how the search engine has recently changed the image search results of “CEO” to reflect a larger proportion of woman than previously presented. They question if this should have taken place even though in 2018 only 5% of fortune 500 companies CEOs were woman (Atkins, 2019).
Representation Bias
If a sample set is too small or under-represented then the input received will not be accurate. A perfect example of this would be from Buolamwini (2016) where upon trying to use open source facial tracking software, she found that the software itself would not detect her black face without the aid of a white face mask. Representation bias would be the cause of this but can take hold at various points throughout the algorithmic method. One such place would be where the input samples feed to the algorithm. If the algorithm is not input with faces of black individuals then the algorithm will not know of a black face existing and thus not register it.
Measurement Bias
Proxies for ideal features are often used due to their availability in place of perfectly empirical data; false positives be can misinterpreted into algorithmic methods which ultimately then leads to measurement bias. O'Neil (2018 pp. 50-68) describes how the leading measurement for college statistics, U.S. News, provides tables for the whole country which often are gamed by colleges to promote them up the table. They do so by exploiting proxies which the algorithm interprets equally, as opposed to more quantifiable but harder to manipulate data. An example of this would be how Texas Christian University (TCU) heavily invested in sports facilities, ultimately improving its sports scoring and due to the popularity of its teams, saw a 30% spike in undergraduate applications over two years. The US News algorithm favours applications as a data point and therefore progressed TCU 37 places up the table in just 7 years. This is all without considering the actual academic capability of the university itself.
Aggregation Bias
Suresh and Guttag (2019) propose that aggregation bias occurs when “one-size-fit-all” methodologies are applied to algorithm modelling. Such as when an algorithm is designed with the assumption that the users are homogeneous in their needs. The authors provide the example of how it is known that diabetes patients differ in their degrees of complication when considering their ethnicity and gender, but that these actual complications go further in subpopulations. Knowing this it would then be ill advised to design an algorithm that tried to best suit any of the groups needs, even if they were equally represented within the algorithms initial model. This would only be advisable if every single need was collected and accurately weighed amongst one another.
Evaluation Bias
When an algorithm does not represent the target population in accordance with evaluation and benchmarking data, evaluation bias occurs. Going back to the Buolamwini (2016) example where facial recognition software was not detecting her black face, it can also be said that evaluation bias was also taking place alongside representation bias. This could have happened if the input population for the software represented a minority of the total users, resulting in a total fractional error rate overall, but a persistent error none the less. Other investigations found data correlating to this where only 7.4% of the total benchmark data sets input into the recognition algorithm were black female faces (Buolamwini and Gebru, 2018).
With the addition of Suresh and Guttag’s (2019) framework for understanding the source of bias within algorithms and machine learning technologies, clarity and perspective can be achieved when analysing examples of bias. Without such an analysis, comprehension of the 300+ types of different biases can be extremely difficult, especially while also trying to manage one's own subconscious biases.