Artificial intelligence: meeting the Public Sector Equality Duty (PSED)

Public bodies need to consider the PSED from the very start when thinking about whether to use AI. The PSED also applies to any AI systems that public bodies are already using or that others may be developing or using on their behalf.

The PSED has two parts:

• the general duty applies to public authorities and organisations carrying out public functions
• specific duties apply only to public authorities named (or listed) in specific duties regulations

There are different specific duties for England, Scotland and Wales but the general duty applies to all three nations.

The general duty requires public authorities and organisations carrying out public functions to have due regard to the need to:

1. eliminate unlawful discrimination harassment and victimisation and any other conduct that is prohibited by the Equality Act 2010
2. advance equality of opportunity between people who share a protected characteristic and those who do not share it
3. foster good relations between people who share a particular protected characteristic and those who do not share it

Organisations subject to the general duty in England, Scotland and Wales must meet the three needs or aims listed above. They must think about how their actions affect equality, good relations, the need to eliminate discrimination and how they can contribute to a more equal society through their day-to-day work. This means, for example, they must consider and review how they are building equality into:

• decision-making
• internal and external policies
• procuring goods and services
• the services they provide
• employment policies and practices (for public bodies only)

Case study: indirect discrimination

A public body uses a new online portal to help it decide how to allocate grants for community projects. When the public body monitors how fairly it has made decisions it finds that it has proportionately turned down more grant applications for Bangladeshi and Pakistani community projects than for other groups. The reasons for this are not immediately clear. On further inspection, it seems that the algorithm used on the portal is identifying certain postcode areas as posing a greater risk of project failure than other areas and that these particular postcode areas have higher proportions of Bangladeshi and Pakistani residents. This may amount to indirect discrimination because the risk assessment undertaken by the AI applies to everybody but disadvantages a group of people who share a protected characteristic without a justifiable reason. It also conflicts with the underlying intentions of the grants programme.

Case study: advancing equality

A health provider develops new software to help it to identify people at risk of missing medical appointments. The system identifies that some disabled people and people who speak different languages, are more at risk of missing their appointments. The system makes sure that the people it identifies receive information in the format they need, such as different languages, braille, or easy read. This helps to make sure that people are less likely to miss their appointments, meaning they get the help they need whilst reducing costly missed appointments.

Case study: fostering good relations

A police force uses AI to assess and predict local crime patterns. This helps it to manage police resources effectively and plan interventions proactively. The data the AI system uses includes monitoring race and disability hate crimes. This police force uses the intelligence generated from the AI to prioritise work with other agencies to improve levels of reporting and prosecution of hate crime offences, and to tackle prejudice and promote understanding.

Certain listed public bodies must also follow the specific duties. These help public bodies improve their performance of the general duty. There are separate specific duties that apply in England, Scotland and Wales.

The specific duties that are particularly relevant to AI are likely to be those relating to:

• assessing equality impact (as applies in Scotland and Wales)
• procurement and commissioning (as applies in Scotland and Wales)
• setting equality objectives and publishing information to show compliance with the general duty (equality outcomes in Scotland)

Workforce-related obligations are also likely to be relevant for any public bodies using AI in their employment.

For a full description of the specific duties that apply in each country, see chapter 6 of the following guides:

• technical guidance on the PSED in England
• technical guidance on the PSED in Wales
• technical guidance on the PSED in Scotland

These guides also contain information about which bodies must comply with the specific duties.