Common-good organizations have a clear mission: They work for the benefit of society, whether in social work, environmental protection, or supporting marginalized groups. For example, “Paulinenpflege Winnenden” runs a facility for youth and disability assistance, “Grüne Erde” sells ecological and sustainable products, and the association of therapy clinics “Ludwigsmühle” helps people with addictions. These organizations’ structures and actions are designed to prioritize social goals. A formal definition (in German) of these enterprises states that their business activities are based on common-good goals, they reinvest most of their profits to achieve social objectives, and their structures promote participation and social justice. But what happens when these common-good goals do not align with the digital tools they use?

In recent years, common-good organizations have also started using large foundation models. Language models like GPT and products built on them, such as ChatGPT or Co-Pilot, are increasingly being used, for instance, in the knowledge management of a large humanitarian organization (Link in German). These models, often developed by big tech companies, offer various advantages: organizations can increase their efficiency through support with routine tasks like documentation and reporting, and the ability of the models to process and analyze large amounts of data in a short time is useful in many work contexts – for example, evaluating donation campaigns, climate data, or patient information. However, they are based on training data that often do not reflect the values essential to common-good organizations, such as fairness, equality, diversity, and the protection of human rights.

Biased training data and its impact on foundation models

Training data – the data foundations of these models – are often large datasets sourced from the internet through automated processes like web scraping. These training data are a key reason why the models are so powerful. But the crucial question is: powerful for whom? Numerous examples show that the models not only reproduce existing biases and stereotypes but even amplify them: from cases of gender-based violence in generative AI models, to gender bias in letters of recommendation generated by language models, to racist advice and stereotypical portrayals of doctors and other professions. This data mostly reflects Western and patriarchal perspectives and values. Biases are thus inevitable: the one-sided data foundation can lead the models to favor or disadvantage certain groups, genders, or viewpoints, failing to adequately represent the diversity and complexity of society, as the numerous examples strikingly illustrate.

A faulty foundation leads to flawed outcomes

Foundation models serve as the foundation for many different AI applications, whether for chatbots, translation, or knowledge management tools. If the foundation of a house is skewed, the walls built on it will be unstable. The same is true for AI models: if the training data already contain biases and errors, these will carry over to the applications built on these models. For instance, if a common-good organization uses a language model based on racist training data for a chatbot providing advice to migrants, the consequences could be problematic.

The decisions and responses of the model would certainly not meet the ethical standards or the specific needs of the organizations and the people affected. The examples above also highlight how vulnerable certain – particularly marginalized – groups and individuals can become when AI-generated predictions or evaluations box them into categories. A good example is the use of AI in human resources, where algorithms, due to biased training data, may systematically disadvantage certain applicants – for instance, women with migrant backgrounds. This can happen when the model incorrectly assumes that people from certain countries or with certain names are less suitable for specific positions. In working with migrants, biases embedded in the models can also lead to people being disadvantaged due to stereotypical assessments – for example, being passed over for housing assignments or misclassified in language courses. Such decisions are often based on biased data and entrench existing inequalities rather than reducing them.

Values-based models: a better choice for common-good oriented organizations?

Common-good oriented organizations are increasingly relying on AI foundation models without considering the values embedded in the underlying training data. They should think about choosing AI models whose training data align better with their own values. It is in their interest to use technology that is not only efficient but also ethically sound – fair and inclusive.

Alternative approaches should also be made more transparent. After all, having a choice gives us as customers and users power. And a value-based choice reinforces the credibility of common-good enterprises and strengthens their impact.

Our study: a close analysis of training data

To address these challenges, at reframe[Tech], we are conducting a study that closely examines the training data of foundation models. We are collaborating with Professor Anne L. Washington from New York University, currently a fellow at Stanford University and author of “Ethical Data Science: Prediction in the Public Interest”, to develop criteria for evaluating the data foundations of foundation models. This study will be published in early 2025.

Common-good organizations have the opportunity to represent their values in the digital space as well – and we want to help them do so responsibly. Just like a stable foundation ensures the security of an entire house, the data foundations of AI models must also be stable and value-based so that the applications built on them meet the ethical standards of the organizations that use them.

Stay informed!

For continuous updates and to engage in discussions on this topic, follow our LinkedIn account or feel free to contact me at teresa.staiger@bertelsmann-stiftung.de.