Government Scraps Immigration “Streaming Tool” before Judicial Review

6 August 2020 by

In response to a legal challenge brought by the Joint Council for the Welfare of Immigrants (JCWI), the Home Office has scrapped an algorithm used for sorting visa applications. Represented by Foxglove, a legal non-profit specialising in data privacy law, JCWI launched judicial review proceedings,, arguing that the algorithmic tool was unlawful on the grounds that it was discriminatory under the Equality Act 2010 and irrational under common law. 

In a letter to Foxglove from 3rd August on behalf of the Secretary of State for the Home Department (SSHD), the Government Legal Department stated that it would stop using the algorithm, known as the “streaming tool”, “pending a redesign of the process and way in which visa applications are allocated for decision making”. The Department denied that the tool was discriminatory. During the redesign, visa application decisions would be made “by reference to person-centric attributes… and nationality will not be taken into account”. 

The “streaming tool” was an algorithmic system designed to categorise visa applications with reference to how much scrutiny each application needed. It would assign an application a red, amber, or green rating: red indicated that the application’s case worker ought to spend more time applying scrutiny, and would have to justify approving the application to a more senior officer. Applications with a red rating were much less likely to be successful than those rated green, with around 99.5% of green being successful but only 48.59% of red. 

The exact weighting of the numerous factors that contributed to the streaming tool’s decision making are not known, as the architecture of the algorithm was not revealed. However, in a letter to Foxglove, the SSHD revealed that “nationality is one of the relevant factors used by the streaming tool”. Certain nationalities are identified in the Equality Act Nationality Risk Assessment (EANRA) as “suspect”. A visa application coming from someone whose nationality was identified in the EANRA would be automatically given a red rating. An applicant’s nationality, even if not on the EANRA “suspect” list, could still, in conjunction with other factors, contribute to the awarding of a red or amber rating.

Nationality is protected from discrimination under Section 4 of the Equality Act. However, the Equality Act does allow for enhanced scrutiny of visa applications on the basis of nationality if prescribed by a Ministerial Authorisation issued under section Schedule 3 of the EA. The use of the streaming tool was justified with reference to the Ministerial Authorisation, as its only authorised use was to signify the need for a “more rigorous examination” of the application.

The Ministerial Authority which legitimised the streaming tool’s categorisation by nationality sets out various routes by which a specific nationality can be placed on the EANRA “suspect” list, most notably a nationality being associated with a high number of “adverse events”. These can include unauthorised behaviours (over-staying, working, etc.). Adverse events also include having a visa application refused. Given that red ratings were typically refused at a higher rate than other ratings, this risked creating a vicious cycle where certain nationalities would be locked onto the EANRA “suspect” list.

Foxglove argued that the use of the streaming tool was discriminatory and irrational. The streaming tool’s only authorised function was to classify applications in relation to required caseworker scrutiny, and not to contribute to decision making. Foxglove held that the ratings materially contributed to the decision making process. They suggested that rating an application red would create confirmation bias, leading case workers to rate evidence contributing negatively to the application more highly than positive evidence. This, they suggested, is evidenced in the difference in success rates between red rated and green rated applications. Furthermore, Foxglove cites a report from the Independent Chief Inspector of Borders and Immigration from 2017 which states that the streaming tool had become a “de facto decision-making tool”. 

Both the confirmation bias and the report make clear how the streaming tool was used beyond its authorised bounds. As nationality was a significant factor in the streaming tool’s weighting (in many cases, the significant factor), its use was argued to be illegal under section 4 of the Equality Act. 

The vicious circle present in the streaming tool “produce[d] substantive results which [were] irrational”. Because visa application refusals were considered to be adverse events, and those same adverse events fed into the algorithm’s decision making, certain nationalities were locked onto the EANRA “suspect” list. This further increased the number of adverse events associated with that nationality, in turn contributing to its position on the EANRA list. As such, the algorithm would class applications as high risk merely because it had done so in the past. Foxglove argued that this constituted irrationality. 

The function of the streaming tool highlights a wider debate surrounding the use of reinforcement learning algorithms and AI in government. Algorithms that feed their own results back into their learning processes, like the streaming tool and other algorithms relying on reinforcement learning, often end up shaping their own learning environments and entrenching biases. This risks manifesting in discriminatory ways. 

While the streaming tool was shelved before a judicial review could be conducted, the Foxglove/JCWI case could prove to be an important referent as more public services use algorithms in their functioning. Foxglove also argued that the government had failed to undertake the Data Protection Impact Assessment required for the use of the streaming tool. The Home Office has committed to a fast redesign, intending to complete it by the latest 30th October 2020.

Welcome to the UKHRB


This blog is run by 1 Crown Office Row barristers' chambers. Subscribe for free updates here. The blog's editorial team is:
Commissioning Editor: Jonathan Metzer
Editorial Team: Rosalind English
Angus McCullough QC David Hart QC
Martin Downs
Jim Duffy

Free email updates


Enter your email address to subscribe to this blog for free and receive weekly notifications of new posts by email.

Subscribe

Categories


Disclaimer


This blog is maintained for information purposes only. It is not intended to be a source of legal advice and must not be relied upon as such. Blog posts reflect the views and opinions of their individual authors, not of chambers as a whole.

Our privacy policy can be found on our ‘subscribe’ page or by clicking here.

Tags


Aarhus Abortion Abu Qatada Abuse Access to justice adoption ALBA Al Qaeda animal rights anonymity Article 1 Protocol 1 Article 2 article 3 Article 4 article 5 Article 6 Article 8 Article 9 article 10 Article 11 article 13 Article 14 Artificial Intelligence Asbestos assisted suicide asylum Australia autism benefits Bill of Rights biotechnology blogging Bloody Sunday brexit Bribery Catholicism Chagos Islanders Children children's rights China christianity citizenship civil liberties campaigners climate change clinical negligence Coercion common law confidentiality consent conservation constitution contempt of court Control orders Copyright coronavirus costs Court of Protection crime Cybersecurity Damages data protection death penalty defamation deportation deprivation of liberty Detention disability disclosure Discrimination disease divorce DNA domestic violence duty of care ECHR ECtHR Education election Employment Environment Equality Act Ethiopia EU EU Charter of Fundamental Rights EU costs EU law European Court of Justice evidence extradition extraordinary rendition Family Fertility FGM Finance foreign criminals foreign office France freedom of assembly Freedom of Expression freedom of information freedom of speech Gay marriage Gaza genetics Germany Google Grenfell Health HIV home office Housing HRLA human rights Human Rights Act human rights news Huntington's Disease immigration India Indonesia injunction Inquests international law internet Inuit Iran Iraq Ireland Islam Israel Italy IVF Japan Judaism judicial review jury trial JUSTICE Justice and Security Bill Law Pod UK legal aid Leveson Inquiry LGBTQ Rights liability Libel Liberty Libya Lithuania local authorities marriage mental capacity Mental Health military Ministry of Justice modern slavery music Muslim nationality national security NHS Northern Ireland nuclear challenges Obituary ouster clauses parental rights parliamentary expenses scandal patents Pensions Personal Injury Piracy Plagiarism planning Poland Police Politics pollution press Prisoners Prisons privacy Professional Discipline Property proportionality Protection of Freedoms Bill Protest Public/Private public access public authorities public inquiries rehabilitation Reith Lectures Religion RightsInfo right to die right to family life Right to Privacy right to swim riots Roma Romania Round Up Royals Russia Saudi Arabia Scotland secrecy secret justice sexual offence Sikhism Smoking social media South Africa Spain special advocates Sports Standing statelessness stop and search Strasbourg Supreme Court Supreme Court of Canada surrogacy surveillance Syria Tax technology Terrorism tort Torture travel treaty TTIP Turkey UK Ukraine USA US Supreme Court vicarious liability Wales War Crimes Wars Welfare Western Sahara Whistleblowing Wikileaks wind farms WomenInLaw YearInReview Zimbabwe
%d bloggers like this: