Keystone Projects

Despite known technical limitations (e.g. biases, privacy leaks, poor reasoning, lack of explainability) Large Language Models (LLMs), are being rapidly adopted without forethought for the repercussions of doing so. In many sectors negative consequences might simply be inconvenient but in sensitive and safety-critical domains, such as healthcare and the law, ramifications of implementing poorly understood, or untested and black box systems, are likely to result in life changing implications for patients, litigants, and other stakeholders. To emphasise the very real risks the public are facing, UK judges are already allowed to utilise LLMs to summarise court cases; what if the LLM gets the temporal ordering of events wrong, or what about LLMs reinforcing racial biases already found in parole decisions? On the medical side we know that public medical question answering services are being rolled out, which could be giving inappropriate information due to lack of factuality and inherent biases. With few expectations on industry developers, information regarding models and training datasets is scant, and responsible research and innovation has taken low priority but this is expected to change due to legislation and public pressure.

Our vision addresses the socio-technical limitations of LLMs that challenge their responsible and trustworthy use, particularly in the context of medical and legal use cases. Our goal is two-fold: Firstly to create an extensive evaluation benchmark (including suitable novel criteria, metrics and tasks) for assessing the limitations of LLMs in real world settings, enabling our standards and policy partners to implement responsible regulations, and industry and third sector partners to robustly assess their systems. The second part of our vision is to devise novel mitigating solutions based on new machine learning methodology, informed by expertise in law, ethics and healthcare, via co-creation with domain experts, that can be incorporated in products and services. The combination of these two core research strands will help enhance capabilities of LLMs for responsible research and innovation, thus allowing society to reap the benefits that LLM based systems promise for many sectors of the economy, while ensuring the prevention of enormous harms.

Probabilistic systems supported by AI (‘Probabilistic AI’), such as facial recognition, predictive tools and large language models, are being introduced at pace into policing, intelligence, probation and wider criminal justice contexts, potentially bringing measurable benefits. But a key problem for responsible AI is that the uncertain – or probable – nature of outputs is often obscured and/or isinterpreted. Challenges to achieving ‘responsible’ AI include data quality, sensitivity and uncertainty. Attention must be paid to chaining of systems and cumulative effects of AI systems feeding each other. Our ambition is for deployment of Probabilistic AI within law enforcement to seek to achieve one coherent system, with justice and responsibility at its heart. We will articulate what ‘responsible’ means for Probabilistic AI across the law enforcement landscape (intelligence, investigation, prosecution, sentencing, probation, oversight), determining technical, operational, legal, ethical and societal implications of the development, deployment and governance of ‘responsible’ AI. To realise our vision, we will draw on multiple perspectives, including from law enforcement and commercial partners, applying lessons from prior developments. We will use innovative research techniques such as a mock trial involving AI outputs as evidence. Our main output will be a future-oriented framework for responsible AI in law enforcement.

The Participatory Harm Auditing Workbenches and Methodologies (PHAWM) project brings together 25 researchers from seven leading UK universities with 23 partner organisations. The University of Glasgow will lead the consortium, with support from colleagues at the Universities of Edinburgh, Sheffield, Stirling, Strathclyde, York and King’s College London. Together, they will develop new methods for maximising the potential benefits of predictive and generative AI while minimising their potential for harm arising from bias and ‘hallucinations’, where AI tools present false or invented information as fact. The project will pioneer participatory AI auditing, where non-experts including regulators, end-users and people likely to be affected by decisions made by AI systems will play a role in ensuring that those systems provide fair and reliable outputs. The project will develop new tools to support the auditing process in partnership with relevant stakeholders, focusing on four key use cases for predictive and generative AI, and create new training resources to help encourage widespread adoption of the tools. The predictive AI use cases in the research will focus on health and media content, analysing data sets for predicting hospital readmissions and assessing child attachment for potential bias, and examining fairness in search engines and hate speech detection on social media. In the generative AI use cases, the project will look at cultural heritage and collaborative content generation. It will explore the potential of AI to deepen understanding of historical materials without misrepresentation or bias, and how AI could be used to write accurate Wikipedia articles in under-represented languages without contributing to the spread of misinformation.