Risks of Discrimination through the Use of Algorithms

Algorithms and extensive data sets are increasingly involved in decisions that not only have trivial consequences for people, but also influence their way of living and personality development to a larger extent. Algorithms produce conclusions and outcomes that are used by human decision-makers as an information basis for their decisions, or the implementation of decision-making rules is completely delegated to algorithms or the computer systems containing them.

Apart from some early analyses of bias and unequal treatment through the use of computer systems (Friedman & Nissenbaum 1996; Bruce & Adam 1989), the risks of discrimination in connection with information and communication technologies have only been comprehensively addressed in this decade.

This study takes a problem-oriented view on the consequences of the use of algorithms for the differentiation of individuals. The study focuses on algorithms that are used to differentiate individuals with regard to differentiated information, products, services, payments, positions, etc. Such differentiations can lead to unjustified unequal treatment or discrimination.

It is therefore a societal task to define what unequal treatment is considered unjustified and to regulate unjustified unequal treatment. This task also concerns the forms of differentiation that are put into practice with the use of algorithms.

Read full study published by the German Federal Anti-Discrimination Agency.