Government Scraps Immigration “Streaming Tool” before Judicial Review

6 August 2020 by

In response to a legal challenge brought by the Joint Council for the Welfare of Immigrants (JCWI), the Home Office has scrapped an algorithm used for sorting visa applications. Represented by Foxglove, a legal non-profit specialising in data privacy law, JCWI launched judicial review proceedings,, arguing that the algorithmic tool was unlawful on the grounds that it was discriminatory under the Equality Act 2010 and irrational under common law. 

In a letter to Foxglove from 3rd August on behalf of the Secretary of State for the Home Department (SSHD), the Government Legal Department stated that it would stop using the algorithm, known as the “streaming tool”, “pending a redesign of the process and way in which visa applications are allocated for decision making”. The Department denied that the tool was discriminatory. During the redesign, visa application decisions would be made “by reference to person-centric attributes… and nationality will not be taken into account”. 

The “streaming tool” was an algorithmic system designed to categorise visa applications with reference to how much scrutiny each application needed. It would assign an application a red, amber, or green rating: red indicated that the application’s case worker ought to spend more time applying scrutiny, and would have to justify approving the application to a more senior officer. Applications with a red rating were much less likely to be successful than those rated green, with around 99.5% of green being successful but only 48.59% of red. 

The exact weighting of the numerous factors that contributed to the streaming tool’s decision making are not known, as the architecture of the algorithm was not revealed. However, in a letter to Foxglove, the SSHD revealed that “nationality is one of the relevant factors used by the streaming tool”. Certain nationalities are identified in the Equality Act Nationality Risk Assessment (EANRA) as “suspect”. A visa application coming from someone whose nationality was identified in the EANRA would be automatically given a red rating. An applicant’s nationality, even if not on the EANRA “suspect” list, could still, in conjunction with other factors, contribute to the awarding of a red or amber rating.

Nationality is protected from discrimination under Section 4 of the Equality Act. However, the Equality Act does allow for enhanced scrutiny of visa applications on the basis of nationality if prescribed by a Ministerial Authorisation issued under section Schedule 3 of the EA. The use of the streaming tool was justified with reference to the Ministerial Authorisation, as its only authorised use was to signify the need for a “more rigorous examination” of the application.

The Ministerial Authority which legitimised the streaming tool’s categorisation by nationality sets out various routes by which a specific nationality can be placed on the EANRA “suspect” list, most notably a nationality being associated with a high number of “adverse events”. These can include unauthorised behaviours (over-staying, working, etc.). Adverse events also include having a visa application refused. Given that red ratings were typically refused at a higher rate than other ratings, this risked creating a vicious cycle where certain nationalities would be locked onto the EANRA “suspect” list.

Foxglove argued that the use of the streaming tool was discriminatory and irrational. The streaming tool’s only authorised function was to classify applications in relation to required caseworker scrutiny, and not to contribute to decision making. Foxglove held that the ratings materially contributed to the decision making process. They suggested that rating an application red would create confirmation bias, leading case workers to rate evidence contributing negatively to the application more highly than positive evidence. This, they suggested, is evidenced in the difference in success rates between red rated and green rated applications. Furthermore, Foxglove cites a report from the Independent Chief Inspector of Borders and Immigration from 2017 which states that the streaming tool had become a “de facto decision-making tool”. 

Both the confirmation bias and the report make clear how the streaming tool was used beyond its authorised bounds. As nationality was a significant factor in the streaming tool’s weighting (in many cases, the significant factor), its use was argued to be illegal under section 4 of the Equality Act. 

The vicious circle present in the streaming tool “produce[d] substantive results which [were] irrational”. Because visa application refusals were considered to be adverse events, and those same adverse events fed into the algorithm’s decision making, certain nationalities were locked onto the EANRA “suspect” list. This further increased the number of adverse events associated with that nationality, in turn contributing to its position on the EANRA list. As such, the algorithm would class applications as high risk merely because it had done so in the past. Foxglove argued that this constituted irrationality. 

The function of the streaming tool highlights a wider debate surrounding the use of reinforcement learning algorithms and AI in government. Algorithms that feed their own results back into their learning processes, like the streaming tool and other algorithms relying on reinforcement learning, often end up shaping their own learning environments and entrenching biases. This risks manifesting in discriminatory ways. 

While the streaming tool was shelved before a judicial review could be conducted, the Foxglove/JCWI case could prove to be an important referent as more public services use algorithms in their functioning. Foxglove also argued that the government had failed to undertake the Data Protection Impact Assessment required for the use of the streaming tool. The Home Office has committed to a fast redesign, intending to complete it by the latest 30th October 2020.

Welcome to the UKHRB


This blog is run by 1 Crown Office Row barristers' chambers. Subscribe for free updates here. The blog's editorial team is:
Commissioning Editor: Jonathan Metzer
Editorial Team: Rosalind English
Angus McCullough QC David Hart QC
Martin Downs
Jim Duffy

Free email updates


Enter your email address to subscribe to this blog for free and receive weekly notifications of new posts by email.

Subscribe

Categories


Tags


Aarhus Abortion Abu Qatada Abuse Access to justice adoption AI air pollution air travel ALBA Allergy Al Qaeda Amnesty International animal rights Animals anonymity Article 1 Protocol 1 Article 2 article 3 Article 4 article 5 Article 6 Article 8 Article 9 article 10 Article 11 article 13 Article 14 article 263 TFEU Artificial Intelligence Asbestos Assange assisted suicide asylum asylum seekers Australia autism badgers benefits Bill of Rights biotechnology blogging Bloody Sunday brexit Bribery British Waterways Board Catholic Church Catholicism Chagos Islanders Charter of Fundamental Rights child protection Children children's rights China christianity citizenship civil liberties campaigners civil partnerships climate change clinical negligence closed material procedure Coercion Commission on a Bill of Rights common law communications competition confidentiality consent conservation constitution contact order contact tracing contempt of court Control orders Copyright coronavirus coronavirus act 2020 costs costs budgets Court of Protection covid crime criminal law Cybersecurity Damages data protection death penalty defamation DEFRA deportation deprivation of liberty derogations Detention Dignitas diplomacy disability disclosure Discrimination disease divorce DNA domestic violence duty of care ECHR ECtHR Education election Employment Environment Equality Act Equality Act 2010 Ethiopia EU EU Charter of Fundamental Rights EU costs EU law European Convention on Human Rights European Court of Human Rights European Court of Justice evidence extradition extraordinary rendition Facebook Facial Recognition Family Fatal Accidents Fertility FGM Finance foreign criminals foreign office foreign policy France freedom of assembly Freedom of Expression freedom of information freedom of speech Gay marriage gay rights Gaza Gender genetics Germany Google Grenfell Gun Control Health HIV home office Housing HRLA human rights Human Rights Act human rights news Human Rights Watch Huntington's Disease immigration India Indonesia injunction Inquests insurance international law internet inuit Iran Iraq Ireland islam Israel Italy IVF ivory ban Japan joint enterprise judaism judicial review Judicial Review reform Julian Assange jury trial JUSTICE Justice and Security Bill Law Pod UK legal aid legal aid cuts Leveson Inquiry lgbtq liability Libel Liberty Libya lisbon treaty Lithuania local authorities marriage Media and Censorship mental capacity Mental Capacity Act Mental Health military Ministry of Justice modern slavery morocco murder music Muslim nationality national security naturism neuroscience NHS Northern Ireland nuclear challenges nuisance Obituary parental rights parliamentary expenses scandal patents Pensions Personal Injury physician assisted death Piracy Plagiarism planning planning system Poland Police Politics Pope press prison Prisoners prisoner votes Prisons privacy Professional Discipline Property proportionality prosecutions Protection of Freedoms Bill Protest Public/Private public access public authorities public inquiries quarantine Radicalisation rehabilitation Reith Lectures Religion RightsInfo right to die right to family life Right to Privacy right to swim riots Roma Romania round-up Round Up Royals Russia saudi arabia Scotland secrecy secret justice Secret trials sexual offence shamima begum Sikhism Smoking social media social workers South Africa Spain special advocates Sports Standing starvation statelessness stem cells stop and search Strasbourg super injunctions Supreme Court Supreme Court of Canada surrogacy surveillance sweatshops Syria Tax technology Terrorism tort Torture travel treason treaty accession trial by jury TTIP Turkey Twitter UK Ukraine universal credit universal jurisdiction unlawful detention USA US Supreme Court vicarious liability Wales War Crimes Wars Welfare Western Sahara Whistleblowing Wikileaks wildlife wind farms WomenInLaw Worboys wrongful birth YearInReview Zimbabwe

Disclaimer


This blog is maintained for information purposes only. It is not intended to be a source of legal advice and must not be relied upon as such. Blog posts reflect the views and opinions of their individual authors, not of chambers as a whole.

Our privacy policy can be found on our ‘subscribe’ page or by clicking here.

%d bloggers like this: