This post was first published on the Aston Institute for Forensic Linguistics blog 27 November 2025

In 1986 Donald Norman and Stephen Draper published the influential edited volume User Centred System Design, a collection of essays that set the agenda for software development and the field of Human-Computer Interaction throughout the 1990s and into the twenty-first century.  In his own chapter in the volume, Norman writes that “user-centred design emphasizes that the purpose of the system is to serve the user not to use a specific technology, not to be an elegant piece of programming.” (Norman & Draper, 1986, p61). As in many forensic sciences, in forensic authorship analysis there has been a rush to develop and adopt the latest specific machine learning technologies such as transformer deep learning architectures, adversarial learning, and Large Language Models (LLMs), which are often colloquially referred to as Artificial Intelligence (AI) systems.  These AI systems process and build models based in enormous amounts of data, often finding associations between and within enormous collections of variables, the scale of which means that these associations are sometimes uninterpretable by traditional statistical means and are certainly beyond the comprehension of human users of these systems.  The demonstrable power and accuracy of these systems in experimental datasets is often only matched by their opacity, leading the human analyst to bewilderment at specific outcomes, and crucially with no way of understanding how the outcome was arrived at.  In the excitement of new and powerful technological potential, Norman’s principle of serving the user seems to have been left behind.

In most jurisdictions AI produced forensic evidence is current not admitted in court cases.  One dilemma that forensic applications of AI systems can face, is illustrated by considering two sets of studies:

• Basu et al (2022), Basu et al (2023) and Bali et al (2024) set out a series of mock-judge and mock-jury studies making use of an AI forensic speech science system known as E³FS³ (Weber et al, 2022), a forensic-voice-comparison system based on state-of-the-art automatic-speaker-recognition technology. E³FS³ was developed by to comply with modern principles for evaluation of evidence, i.e., that methods be transparent and reproducible, logically correct, resistant to cognitive bias, and calibrated and validated under casework conditions (Morrison, 2022).  The Basu et al (2022, 2023) and Bali et al (2024) studies compared the perceived strength of evidence of the human listeners vs the strength of evidence output by the system and did so with the human listeners acting alone (as a mock individual judges) and working in groups (as mock jurors).  The results of these studies are startling:  E³FS³outperformed all individual listeners and 78% of the groups of collaborating listeners.  When the output of the E³FS³ system was provided to assist human listeners in making their judgement, the performance of listeners provided with the output of the forensic-voice-comparison system was better than the human performance alone – the system improved human performance; however, the performance of listeners provided with the output of the forensic-voice-comparison system was worse than that of the voice comparison system without human judgements – the introduction of human listeners degraded the E³FS³ system performance.  These results suggest that an expert witness in court would be required to present, explain and defend such a system in its assessment of the evidence, but that the human jury (or other decision-makers) should not be allowed to listen to the evidence themselves and make their own judgements.  They should accept the evaluation of a well-validated system on the basis of the validation alone.   The system once validated (and calibrated to a particular case) cannot be questioned or confronted – in its use in court it is an ipse dixit machine.

• A second set of studies relates to challenges put forward by the PAN authorship analysis competitions (https://pan.webis.de).   Since the introduction of authorship verification tasks (of various designs) to the PAN competitions, AI and complex computational approaches appeared to have solved the problem such that by 2021 all results in that year’s verification task competition exceeded 93% accuracy (Stamatatos et al., 2022). However, for the 2022 competition, a challenge was introduced relating to a more forensically realistic problem, that of cross-genre or cross-discourse-type analysis drawn from the 100 Idiolects Corpus (Kredens, Heini and Pezik, 2021).  The results of the PAN2022 competition were that the best-in-class systems and models, when applied to the 100 Idiolects Corpus, were reduced in accuracy to rates close to chance.  Stamatatos et al (2022, p11) write that the previous PAN competitions gave “the false impression that authorship verification is an almost solved problem”.  For linguists generally and forensic linguists specifically, the fact that cross-genre analysis would present real challenges to the computational approaches was unsurprising.  For the computational researchers, genre homogeneity / heterogeneity could be characterised as an unrecognised source of bias in the training sets.  In applied AI work it is well acknowledged that understanding such biases are crucial, but this had not come to the fore in the previous PAN authorship competitions.  It is easy to envisage a real forensic case where an advanced authorship analysis system was applied with the validation that it was at least 93% accurate – but where, if the case happened to involve a cross-genre analysis this would create a very real risk of a miscarriage of justice.  This situations demonstrates one risk of relying on system outputs which cannot be cross examined by human experts.  Human expertise and understanding are essential not only for the development of forensic evidence AI systems, but also for their use in specific individual cases in court.

In law, hearsay is excluded from US and European courts on the basis of the 6th Amendment defendant’s right to confront a witness testifying against them, and more or less equivalently, Article 6(3)(d) of the European Convention on Human Rights (ECHR) provides for the right of the accused ‘to examine or have examined witnesses against him’.   Jurs and DeVito (2024) argue strongly that AI systems that lack transparency and explanation and so breach these rights to  confrontation should not be admissible in court. Their proposal to avoid this risk is that “that only sufficiently transparent and explainable systems (open-box AI systems) should be permissible as evidence in courtrooms today” (p597).  They go on to say that a system is explainable “if, when answering queries, it presents information to the user that provides a qualitative understanding of the connection between the input and the output (p614)”.  

The answer to the dilemma of human involvement being both necessary and reducing the efficacy of AI systems is case-by-case explanation of the connection between input and output.  This doesn’t have to be a naïve rejection of the complexity of science.  Ypma et al, 2023 argue that “Triers-of-fact do not need to understand the details of forensic evidence evaluation – they need to be able to trust the experts and the systems they use.” (p5). They point to the complexity of a glass fragment analysis, suggesting a kind of chain of understanding – arguing convincingly that the jury don’t need to know the relationship between the chemical composition of glass fragments and its reflective properties, so long as someone does.

As Ypma et al, (2023) caution, this view of complexity in  forensic science as cannot account for AI generated forensic evidence where a ‘black box’ system is at the heart of the analysis, as this breaks the chain of explanation.

In this instance the development of the technique will involve training a system in an authorship analysis task.  This is likely to involve use of the latest specific technology of LLMs, where input and output conditions are set, and the parameters within the system associate large numbers of input conditions, in large sets of training examples. Coming to the specific case a particular input is set, and the system produces the trained correlating output as its answer.   This approach leaves the jury or decision-maker with no information about consistencies or differences in authorial style and essentially having to trust that the ipse dixit machine has come to a correct conclusion in any specific case. The jurors trust can only rest on the general case validation results rather than the specific case in question, giving a very real possibility of situations parallel to the PAN competitions, where there were hidden biases in the training set turned out to be crucial.

The solution to both the problem of bias in theory free development (the PAN competitions) and humans potentially reducing the accuracy of results (the mock jury findings), has to be a renewed focus on explanation.  Explainable AI (abbreviated to XAI) is a growing field, but it is rare to see advocates of AI taking on a basic user centred design approach.  User personas for a forensic evidence system (for authorship analysis) must include generalist police investigators or crime analysts, expert language analysts, and of course most importantly decision-makers such as jurors or judges in court.  The system must be designed to meet these individual user needs.  We need User Centred Explanation Design and as with User Centred System Design we need to develop methods to understand and document both commonalities and differences between these user groups and we must do this before we start to build any AI forensic science system.

Whilst these end-user analyses remain undone, we can adduce some general features of an explainable AI system for forensic evidence – the purpose of a good explanation will be to allow a human to calibrate trust and understanding in a system.  The end user in each group must be able to say 

• globally – generally the system gets to the right answer in a way that I understand
• in an individual case 
  • I can understand why and when  a system it’s likely to make good decisions and most importantly, when it might make mistakes.  Mistakes can include result with inflated or deflated likelihoods, for example. Where there is a useable explanation such  mistakes doesn’t corrode generalised trust in the system – “The system is wrong, but not nuts! I understand why it says that.”
  • I can the system must provides sufficient evidence of link between inputs and outputs.  This enables me to trust the chain of understanding and to make rational decisions, particularly against contrary independent evidence from other forensic sciences and types of evidence. 

This is may be high bar for explainable AI systems but there are already forms of XAI that begin to meet these criteria.  Until these techniques reach maturity, and until the explanation needs of specific end-users in forensic cases are properly understood and accounted for,  AI produced forensic evidence should not be admitted as evidence in court.