We use cookies to enhance your browsing experience. If you continue to use our website we will take this to mean that you agree to our use of cookies. If you want to find out more, please view our cookie policy. Accept and Hide [x]
In Episode 163, Rosalind English talks to Ariane Adam and Tatiana Kazim of the Public Law Project about automated decision making (ADM) in the public sector, the problems of transparency and automation bias where these decisions affect people’s rights. This interview was held shortly after the House of Lords Justice and Home Affairs Committee published its report on new technologies and the application of the law.
We discuss a number of issues, in particular those that arose in the Post Office “Horizon” accountancy scandal, and the case of R (Eisai Ltd) v National Institute for Health and Clinical Excellence [2008] EWCA Civ 438. The defendant, responsible for appraising clinical benefits and cost-effectiveness of health care interventions, had refused to provide the claimant with a fully executable version of the model it used to assess the cost-effectiveness of the claimant’s drugs. The Court of Appeal held that procedural fairness required release of the fully executable version of the model [66]. It rejected the defendant’s claims that disclosure would undermine confidentiality or be overly costly, noting at [65] that the court should be ‘very slow to allow administrative considerations of this kind to stand in the way of its release’.
The PLP has also published a summary of the JHAC report here.
An Amsterdam Court has ordered Ola (a smartphone-hailing taxi organisation like Uber) to be more transparent about the data it uses as the basis for decisions on suspensions and wage penalties, in a ruling that breaks new ground on the rights of workers subject to algorithmic management.
James Farrarr and Yaseen Aslam, who won the landmark victory in the UK Supreme Court in February, led the action by a group of UK drivers and a Portuguese driver, who bought three separate cases against Ola and Uber seeking fuller access to their personal data.
The following is a summary of the case against Ola taxis. Anton Ekker (assisted by AI expert Jacob Turner, whom we interviewed on Law Pod UK here) represented the drivers. He said that this case was the first time, to his knowledge, that a court had found that workers were subject to automated decision-making (as defined in Article 22 of the GDPR) thus giving them the right to demand human intervention, express their point of view and appeal against the decision.
The Facts
Ola is a company whose parent company is based in Bangalore, India. Ola Cabs is a digital platform that pairs passengers and cab drivers through an app. The claimants are employed as ‘private hire drivers’ (“drivers”) in the United Kingdom. They use the services of Ola through the Ola Driver App and the passengers they transport rely on the Ola Cabs App.
Proceedings are pending in several countries between companies offering services through a digital platform and drivers over whether an employment relationship exists.
By separate requests dated 23 June 2020, the first two claimants requested Ola to disclose their personal data processed by Ola and make it available in a CSV file. The third claimant made an access request on 5 August 2020. Ola provided the claimants with a number of digital files and copies of documents in response to these requests.
Ola has a “Privacy Statement” in which it has included general information about data processing.
All references in this judgment is to the AVG, which is Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of individuals with regard to the processing of personal data and on the free movement of such data (GDPR).
The Court of Appeal, overturning a Divisional Court decision, has found the use of a facial recognition surveillance tool used by South Wales Police to be in breach of Article 8 of the European Convention on Human Rights (ECHR). The case was brought by Liberty on behalf of privacy and civil liberties campaigner Ed Bridges. The appeal was upheld on the basis that the interference with Article 8 of the ECHR, which guarantees a right to privacy and family life, was not “in accordance with law” due to an insufficient legal framework. However, the court found that, had it been in accordance with law, the interference caused by the use of facial recognition technology would not have been disproportionate to the goal of preventing crime. The court also found that Data Protection Impact Assessment (DPIA) was deficient, and that the South Wales Police (SWP), who operated the technology, had not fulfilled their Public Sector Equality Duty.
In response to a legal challenge brought by the Joint Council for the Welfare of Immigrants (JCWI), the Home Office has scrapped an algorithm used for sorting visa applications. Represented by Foxglove, a legal non-profit specialising in data privacy law, JCWI launched judicial review proceedings,, arguing that the algorithmic tool was unlawful on the grounds that it was discriminatory under the Equality Act 2010 and irrational under common law.
In a letter to Foxglove from 3rd August on behalf of the Secretary of State for the Home Department (SSHD), the Government Legal Department stated that it would stop using the algorithm, known as the “streaming tool”, “pending a redesign of the process and way in which visa applications are allocated for decision making”. The Department denied that the tool was discriminatory. During the redesign, visa application decisions would be made “by reference to person-centric attributes… and nationality will not be taken into account”.
Close up of Asian women with hi tech digital technology screen over the eye.
PHG, linked with Cambridge University, provides independent advice and evaluations of biomedical and digital innovations in healthcare. PHG has recently published a series of reports exploring the interpretability of machine learning in this context. The one I will focus on in this post is the report considering the requirements of the GDPR for machine learning in healthcare and medical research by way of transparency, interpretability, or explanation. Links to the other reports are given at the end of this post.
Just a brief summary of machine learning in healthcare (for the detail, go to PHG’s report Machine Learning Landscape).
Machine learning typically denotes “methods that only have task-specific intelligence and lack the broad powers of cognition feared when ‘AI’ is mentioned”. Artificial intelligence (AI) can be defined as “the science and engineering of making computers behave in ways that, until recently, we thought required human intelligence.” We are only beginning to realise the scope of intelligence that is silicone-based, rather than meat-based, in the reductionist words of neurscientist and author Sam Harris. It is important too to grasp the difference between types of programming. As this report puts it,
Machine learningas a programming paradigm differs from classical programming in that machine learning systems are trained rather than explicitly programmed. Classical programming combines rules and data to provide answers. Machine learning combines data and answers to provide the rules
Earlier this month, the Scottish Parliament’s Justice Sub-Committee on Policing published a report which concluded that live facial recognition technology is currently “not fit” for use by Police Scotland.
Police Scotland had initially planned to introduce live facial recognition technology (“the technology”) in 2026. However, this has now been called into question as a result of the report’s findings – that the technology is extremely inaccurate, discriminatory, and ineffective. Not only that, but it also noted that the technology would be a “radical departure” from Police Scotland’s fundamental principle of policing by consent.
In light of the above, the Sub-Committee concluded that there would be “no justifiable basis” for Police Scotland to invest in the technology.
Police Scotland agreed – at least for the time being – and confirmed in the report that they will not introduce the technology at this time. Instead, they will engage in a wider debate with various stakeholders to ensure that the necessary safeguards are in place before introducing it. The Sub-Committee believed that such a debate was essential in order to assess the necessity and accuracy of the technology, as well as the potential impact it could have on people and communities.
The report is undoubtedly significant as it reaffirms that the current state of the technology is ineffective. It therefore strengthens the argument that we should have a much wider debate about the technology before we ever introduce it onto our streets. This is important not only on a practical level but also from a human rights perspective, especially set against the backdrop of the technology’s controversial use elsewhere.
In the latest Henry Brooke Lecture (12th November, hosted by BAILII and Freshfields Bruckhaus Deringer), Supreme Court Justice Lord Sales warned that the growing role of algorithms and artificial intelligence in decision making poses significant legal problems.
He cited as an example a recent case in Singapore. The judge had to decide on mistake in contract – except that the two contracting parties were both algorithms. In that instance the judge was able to identify the human agents behind the programmes, but that will soon not be the case.
The use of algorithms in public sector decision making has
broken through as a hot topic in recent weeks. The Guardian recently ran the “Automating
Poverty” series on the use of algorithms in the welfare state. And on 29
October 2019 it was reported
that the first known legal challenge to the use of algorithms in the UK, this
time by the Home Office, had been launched. It was timely, then, that the
Public Law Project’s annual conference on judicial review trends and forecasts
was themed “Public law and technology”.
Basic tech for lawyers
The conference helpfully opened with a lawyer-friendly run down of algorithms and automation. Dr. Reuben Binns (ICO Postdoctoral Research Fellow in AI) drew a number of useful distinctions.
The first was between rule-based and statisticalmachine learning systems. In rule-based systems, the system is programmed to apply a decision-making tree. The questions asked and the path to a particular outcome, depending on the answers given, can be depicted by way of flow-chart (even if that flow-chart might be very large, involving numerous branches). In contrast, statisticalmachinelearning involves a computer system training itself to spot patterns and correlations in data sets, and to make predictions based on those patterns and correlations. The computer system is first trained on data sets provided by the system designer. Once trained, it can be used to infer information and make predictions based on new data. These systems might be used, for example, to assess the risk of a person re-offending, where the system has been trained on existing data as to re-offending rates. It has long been known that machine-learning systems can be biased, not least because the data on which they are trained is often biased.
The main recommendation in the report is that autonomous power to hurt, destroy or deceive human beings should never be vested in artificial intelligence. The committee calls for the Law Commission to clarify existing liability law and considers whether it will be sufficient when AI systems malfunction or cause harm to users. The authors predict a situation where it is possible to foresee a scenario where AI systems may
malfunction, underperform or otherwise make erroneous decisions which cause harm. In particular, this might happen when an algorithm learns and evolves of its own accord.
The authors of the report confess that it was “not clear” to them or their witnesses whether “new mechanisms for legal liability and redress in such situations are required, or whether existing mechanisms are sufficient”. Their proposals, for securing some sort of prospective safety, echo Isaac Asimov’s three laws for robotics.
A robot may not injure a human being or, through inaction, allow a human being to come to harm.
A robot must obey orders given it by human beings except where such orders would conflict with the First Law.
A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.
But these elaborations of principle may turn out to be merely semantic. The safety regime is not just a question of a few governments and tech companies agreeing on various principles. This is a global problem – and indeed even if Google were to get together with all the other giants in this field, Alibaba, Alphabet, Amazon, Apple, Facebook, Microsoft and Tencent, it may not be able to anticipate the consequences of building machines that can self-improve. Continue reading →
We have just posted a discussion here between 1 Crown Office Row recruit Thomas Beamont and Rosalind English on the reach of Artificial Intelligence into the legal world: click on Episode 10 of our podcast series.
Law Pod UK is freely available for download on iTunes
Artificial intelligence … it’s no longer in the future. It’s with us now.
I posted a review of a book about artificial intelligence in autumn last year. The author’s argument was not that we might find ourselves, some time in the future, subservient to or even enslaved by cool-looking androids from Westworld. His thesis is more disturbing: it’s happening now, and it’s not robots. We are handing over our autonomy to a set of computer instructions called algorithms.
If you remember from my post on that book, I picked out a paragraph that should give pause to any parent urging their offspring to run the gamut of law-school, training contract, pupillage and the never never land of equity partnership or tenancy in today’s competitive legal industry. Yuval Noah Harari suggests that the everything lawyers do now – from the management of company mergers and acquisitions, to deciding on intentionality in negligence or criminal cases – can and will be performed a hundred times more efficiently by computers.
Now here is proof of concept. University College London has just announced the results of the project it gave to its AI researchers, working with a team from the universities of Sheffield and Pennsylvania. Its news website announces that a machine learning algorithm has just analysed, and predicted, “the outcomes of a major international court”:
The judicial decisions of the European Court of Human Rights (ECtHR) have been predicted to 79% accuracy using an artificial intelligence (AI) method.
Not only is God dead, says Israeli professor Yuval Noah Harari, but humanism is on its way out, along with its paraphernalia of human rights instruments and lawyers for their implementation and enforcement. Whilst they and we argue about equality, racism, feminism, discrimination and all the other shibboleths of the humanist era, silicon-based algorithms are quietly taking over the world.
His new book, Homo Deus, is the sequel to Homo Sapiens, reviewed on the UKHRB last year. Sapiens was “a brief history of mankind”, encompassing some seventy thousand years. Homo Deus the future of humankind and whether we are going to survive in our present form, not even for another a thousand years, but for a mere 200 years, given the rise of huge new forces of technology, of data, and of the potential of permissive rather than merely preventative medicine.
We are suddenly showing unprecedented interest in the fate of so-called lower life forms, perhaps because we are about to become one.
Harari’s message in Sapiens was that the success of the human animal rests on one phenomenon: our ability to create fictions, spread them about, believe in them, and then cooperate on an unprecedented scale. These fictions include not only gods, but other ideas we think fundamental to life, such as money, human rights, states and institutions. In Homo Deus he investigates what happens when these mythologies meet the god-like technologies we have created in modern times.
In particular, he scrutinises the rise and current hold of humanism, which he regards as no more secure than the religions it replaced. Humanism is based on the notion of individuality and the fundamental tenet that each and everybody’s feelings and experiences are of equal value, by virtue of being human. Humanism cannot continue as a credible thesis if the concept of individuality is constantly undermined by scientific discoveries, such as the split brain, and pre-conscious brain activity that shows that decisions are not made as a result of conscious will (see the sections on Gazzaniga’s and Kahneman’s experiments in Chapter 8 “The Time Bomb in the Laboratory”).
…once biologists concluded that organisms are algorithms, they dismantled the wall between the organic and inorganic, turned the computer revolution from a purely mechanical affair into a biological cataclysm, and shifted authority from individual networks to networked algorithms.
… The individual will not be crushed by Big Brother; it will disintegrate from within. Today corporations and governments pay homage to my individuality, and promise to provide medicine, education and entertainment customised to my unique needs and wishes. But in order to do so, corporations and governments first need to break me up into biochemical subsystems, monitor these subsystems with ubiquitous sensors and decipher their working with powerful algorithms. In the process, the individual will transpire to be nothing but a religious fantasy.
You don’t need to be a brain scientist to see that lawyers would benefit from a more sophisticated understanding of the human brain. Neuroscientists seek to determine how brain function affects human behaviour, and the system of law regulates how those humans interact with each other. According to a new Royal Society report, lawyers and neuroscientists should work together more.
The report, Neuroscience and the law, argues that neuroscience has a lot to offer the law, for example:
might neuroscience fundamentally change concepts of legal responsibility? Or could aspects of a convicted person’s brain help to determine whether they are at an increased risk of reoffending? Will it ever be possible to use brain scans to ‘read minds’, for instance with the aim of determining whether they are telling the truth, or whether their memories are false?
This blog is maintained for information purposes only. It is not intended to be a source of legal advice and must not be relied upon as such. Blog posts reflect the views and opinions of their individual authors, not of chambers as a whole.
Our privacy policy can be found on our ‘subscribe’ page or by clicking here.
You don’t need to be a brain scientist to see that lawyers would benefit from a more sophisticated understanding of the human brain. Neuroscientists seek to determine how brain function affects human behaviour, and the system of law regulates how those humans interact with each other. According to a new Royal Society report, lawyers and neuroscientists should work together more.
This blog is run by
Recent comments