Is your tech discriminatory?

Employers are increasingly reliant on technology to assist with all kinds of functions – from strengthening security to streamlining recruitment processes.

Most organisations will consider the basic data protection implications of incorporating technology into employee processes, but there are other legal risks that may be relevant but less obvious.

Indirect tech discrimination

Two separate union-backed claims have recently been brought against Uber in relation to facial recognition systems that allegedly indirectly discriminate on the ground of race.

Uber drivers are required to log in by providing a photograph of their face. If the photograph taken does not match the photograph on file, they risk having their account terminated. The Independent Workers Union of Great Britain (IWGB) and App Drivers and Couriers Union (ADCU) each contend that Uber’s facial recognition system generates more inaccurate results when used by black and minority ethnic workers, leaving them more vulnerable to losing their jobs with Uber. The Equality and Human Rights Commission is supporting the case being pursued by the ADCU.

Automated decision-making (ADM)

UK GDPR gives data subjects the right not to be subject to a decision based solely on automated processing, including profiling, if the decision has legal effects or is otherwise significant in its impact – unless certain safeguards are in place.

It’s an area of data protection that has remained relatively under-the-radar, and many organisations routinely declare that ADM is not taking place – without stopping to ask what ADM really means and when or where it might be used. With increased integration of AI into recruitment processes, this is one area where employers may need to revisit the default position of declaring ADM is not applicable.

The ICO is currently consulting on its Employment Practice Guidance and legal practitioners have flagged this as a key aspect that requires updating given the pace of technological change. Employers would benefit from guidance clearly delineating between decisions made solely by automatic means without human intervention and automated processes which merely assist human decision-making – with examples and case studies.

Watch this space

Whilst the tribunal outcomes and updated ICO guidance are all still a little way off, the developments indicate an important direction of travel. As technology becomes increasingly integrated into decision-making, employers need to recognise that their systems – as well as individual members of staff – could be a source of litigation risk.

Contact our employment team for further legal support.