Saturday, 7 September 2013

Barry George case: new insights on the evidence

Jill Dando
Barry George

Our new paper*  "When ‘neutral’ evidence still has probative value: implications from the Barry George Case" (published in the journal Science and Justice) casts doubts on the reasoning in the 2007 Appeal Court judgement that led to the quashing of Barry George's conviction for the shooting to death of TV celebrity Jill Dando.

The paper examines the transcript of the Appeal in the context of new probabilistic research about the probative value of evidence. George's successful appeal was based primarily on the argument that the prosecution's evidence about a particle of firearm discharge residue (FDR) discovered in George's coat pocket, was presented in a way that may have misled the jury. Specifically, the jury in the original trial had heard that the FDR evidence was very unlikely to have been found if Barry George had not fired the gun that killed Jill Dando. Most people would interpret such an assertion as strong evidence in favour of the prosecution case. However, afterwards the same forensic expert concluded that the FDR evidence was just as unlikely to have been discovered if Barry George had  fired the gun. In such a scenario the evidence is considered to be ‘neutral’ - favouring neither the prosecution nor the defence. Hence, the appeal court considered the verdict unsafe and the conviction was quashed. Following the appeal ruling, the FDR was excluded from the jury at George's retrial and he was acquitted.  However, our paper shows that the FDR evidence may not have been neutral after all. 

Formally, the probative value of evidence is captured by a simple probability formula called the likelihood ratio (LR). The LR is the probability of finding the evidence if the prosecution hypothesis is true divided by the probability of finding the evidence if the defence hypothesis is true. Intuitively, if the LR is greater than one then the evidence supports the prosecution hypothesis; if the LR is less than one it supports the defence hypothesis, and if the LR is equals to one (as in the case of the FDR evidence here) then the evidence favours neither and so is 'neutral'.  Accordingly the LR is a commonly recommended method for forensic scientists to use in order to explain the probative value of evidence. However, the new research in the paper shows that the prosecution and defence hypotheses have to be formulated in a certain way in order for the LR to 'work' as expected. Otherwise it is possible, for example, to have evidence whose LR is equal to one but which still has significant probative value.  Our review of the appeal transcript shows that relevant prosecution and defence hypotheses were not properly formulated and, if one were to follow the arguments recorded in the Appeal judgement verbatim, then contrary to the Appeal conclusion, the probative value of the FDR evidence may not have been neutral as was concluded, but rather still supported the prosecution**.

*Full details: Fenton, N. E., D. Berger, D. Lagnado, M. Neil and A. Hsu, (2013). "When ‘neutral’ evidence still has probative value (with implications from the Barry George Case)", Science and Justice, http://dx.doi.org/10.1016/j.scijus.2013.07.002 published online 19 August 2013. For those who do not have full access to the journal, a pre-publication draft of the article can be found here.

** Although the FDR evidence may have been probative after all, we are not in a position to comment on the overall case against Bary George, which others have argued was not particularly strong. Also, it could be argued that even though the FDR evidence was not 'neutral' as assumed in the Appeal, its probative value may not have been as strongly favourable to the prosecution as implied in the original trial; this may have been sufficient in itself to cast doubt on the safety of the conviction.

4 comments:

  1. "Any legal trial seeks to determine whether one or more hypothesis is either true or false."

    Oh no it doesn't! The prosecutor, defence counsel, judge and jury were not there. They have no way of knowing what is true or false. That is why they do not concern themselves with such matters. A trial is a process. The process is operated and it comes up with a verdict.

    From a positive point of view there are two legitimate questions:
    1. In this case was the process followed in a disciplined manner?
    2. In general, is this a process that we can improve on?

    It strikes me that this, highly learned, analysis is really addressing question (2). It is therefore unhelpful only to analyse one trial. There will be other trials where dwelling on probabilistic fine points will increase costs, delay proper acquittal, confuse nontechnical juries and fall athwart other well established principles of the justice process.

    The trial process is something that has huge potential for improvement. Helpful suggestions based on a proper statistical analysis of that process would be welcome.

    ReplyDelete
  2. Anthony

    When we say that "Any legal trial seeks to determine whether one or more hypothesis is either true or false."
    it should be very clear form all our writings that we, of all people, recognise that the lawyers and jury can never KNOW what is true or false. The whole point of the Bayesian approach is that the evidence should enable these people to reduce their uncertainty (subjective belief) in which hypotheses are more likely to be true. In the old days in UK criminal trials the jury were supposed to have narrowed their uncertainty down to a very low (but undefined) level, namely 'beyond resonable doubt'. That no longer applies and juries are now instructed to convict only if they are 'certain' of guilt. Hence, according to the law it is actually technically correct to say that the objective of the trial is to determine if one or more hypothesis is true.

    ReplyDelete
  3. Norman

    I am not a criminal practitioner but I am fairly certain that the word "certain" is deprecated and would get the matter appealed. Judges are to instruct the jury that they must be "sure". In fact, now I check the Bench Book (http://www.judiciary.gov.uk/Resources/JCO/Documents/Training/benchbook_criminal_2010.pdf) I see that I am correct.

    I accept that legal procedure should be advised by statistical theory at some level. However, I also think it is important to look at the process as a whole. When we come up with a new rule of evidence we have to look at how it would influence outcomes across the aggregate of cases, not an isolated judgment. "Outcomes" includes extra use of court and legal resources because of complexity, and society's perceptions.

    I do agree that truth is material in the sense that it would be better to have a process with improved discrimination in punishing the guilty and acquitting the innocent. However, I am very cautious about too much theory. The matrix of evidence in any case is rich and some of it subliminal. Models of reasoning can go so far but we never ever do know who was actually guilty and who not. At the end of the day, I fear (pace Richard Jeffrey), that "It's probabilities all the way down".

    Can you formulate your argument in terms of a Dutch book?

    ReplyDelete
  4. Our book "Risk Assessment and Decision Analysis with Bayesian Networks" (www.bayesianrisk.com) has a relevant Dutch book example starting the middle of Chapter 1, page 24 (Chapter is freely downloadable from the book website).

    Norman

    ReplyDelete