I was happy to be asked
to write a feature for the latest Skeptic magazine picking out skeptical and
science based stuff from the news. It was good fun but as usual I wrote too
much and so I’m reproducing here part of that piece which they didn’t have room
for. If you want to read the rest of what I wrote plus a load of other great
articles then follow this link for details on how to subscribe/send off for one or pick up a magazine at our
next Skeptics in the Pub event. The article follows:-
Leader of the Cornwall Council Tory group, Fiona Ferguson, has resigned her position. The prompt for this has been the rolling out of Voice Risk Analysis (VRA) in her area as a means of determining benefit cheats, specifically people that might be falsely claiming single person’s council tax relief. In these days of austerity you would expect a Conservative councillor to support any means possible to claw back money; so why the impassioned move?
VRA is a lie detector technology developed by Israeli company Nemesysco, that will allegedly help benefit officers determine from a telephone call whether the person is a potential high risk claimant deserving further investigation. Well actually the websites of Digilog - the company with the UK license for the technology - and Capita , that well known repository of public money who employ their tools on behalf of the government, fall over themselves to avoid that term. However if you dig around Capita’s website you will eventually find this sentence … “Lie detection is a difficult process, but Digilog's approach using voice risk analysis ensures a fast and accurate verification of the genuine nature of applicants.” Oops, duck, walk quack etc.
You can understand their reluctance in calling it a lie detector though. Most people outside of the Jeremy Kyle core demographic understand the unreliability of polygraph devices in general. Despite the fact that some agencies such as the FBI and CIA employ them, scientific institutions all around the world have trialled polygraphs in all kinds of situations and found not enough evidence of accuracy or reliability to be able to advise their deployment.
Polygraphs as generally understood measure a range of physiological indicators such as blood pressure, skin conductivity, respiration and so forth to provide stress level comparators which hopefully indicate veracity. Capita claims to be able to do this just by having an operator monitor variations in the voice and speech patterns of the caller. There are immediate limitations apparent even discounting issues such as people with speech impediments or English as a second language.
In 2010 Francisco Lacerdo and Anders Eriksson, respectively professors of Linguistics and Phonetics co-authored a review of the previous fifty years’ worth of evidence of the reliability of this type of technology. Nemesysco’s response was to threaten them with libel claiming that the report was inaccurate. As we all know, the correct method for challenging and progressing science is not to review it in the open but to squash it in the libel courts. Francisco Lacerdo continued to speak out about this technology and I can heartily recommend researching him and finding his blogs and articles on the subject. The original paper, Charlatanry in forensic speech science: A problem to be taken seriously, is still available online.
Cornwall are not the first or only council to deploy VRA, there are others around the country doing so and the Internet is littered with Freedom of Information requests to councils asking them if they use it and how they apply it. Some such as Harrow have tried it, found it ineffective and abandoned it. The Department of Works and Pensions actually commissioned an evaluation trial that ran from 2007 to 2010 in twenty-four different regions. The final sentence of the final report from this reasonably thorough two phase trial was “No further trialling of VRA is planned and on the basis of this evaluation we cannot make any recommendations for its use within benefit processing.”
Just a few months afterwards this more positive but qualitative, evaluation was released by the DWP. It focussed more on how people felt using VRA and how fast staff could work through cases rather than on accuracy. Perhaps this was deemed more important as the system continues to be used to this day in some regions, perhaps one near you.
Ben Goldacre and others have recently argued for a greater use of trials and evidence to guide public spending. To those that agree with this approach the decision to outlay money in rolling out VRA must appear to be a very odd one indeed.
By Patrick Redmond
Leader of the Cornwall Council Tory group, Fiona Ferguson, has resigned her position. The prompt for this has been the rolling out of Voice Risk Analysis (VRA) in her area as a means of determining benefit cheats, specifically people that might be falsely claiming single person’s council tax relief. In these days of austerity you would expect a Conservative councillor to support any means possible to claw back money; so why the impassioned move?
VRA is a lie detector technology developed by Israeli company Nemesysco, that will allegedly help benefit officers determine from a telephone call whether the person is a potential high risk claimant deserving further investigation. Well actually the websites of Digilog - the company with the UK license for the technology - and Capita , that well known repository of public money who employ their tools on behalf of the government, fall over themselves to avoid that term. However if you dig around Capita’s website you will eventually find this sentence … “Lie detection is a difficult process, but Digilog's approach using voice risk analysis ensures a fast and accurate verification of the genuine nature of applicants.” Oops, duck, walk quack etc.
You can understand their reluctance in calling it a lie detector though. Most people outside of the Jeremy Kyle core demographic understand the unreliability of polygraph devices in general. Despite the fact that some agencies such as the FBI and CIA employ them, scientific institutions all around the world have trialled polygraphs in all kinds of situations and found not enough evidence of accuracy or reliability to be able to advise their deployment.
Polygraphs as generally understood measure a range of physiological indicators such as blood pressure, skin conductivity, respiration and so forth to provide stress level comparators which hopefully indicate veracity. Capita claims to be able to do this just by having an operator monitor variations in the voice and speech patterns of the caller. There are immediate limitations apparent even discounting issues such as people with speech impediments or English as a second language.
In 2010 Francisco Lacerdo and Anders Eriksson, respectively professors of Linguistics and Phonetics co-authored a review of the previous fifty years’ worth of evidence of the reliability of this type of technology. Nemesysco’s response was to threaten them with libel claiming that the report was inaccurate. As we all know, the correct method for challenging and progressing science is not to review it in the open but to squash it in the libel courts. Francisco Lacerdo continued to speak out about this technology and I can heartily recommend researching him and finding his blogs and articles on the subject. The original paper, Charlatanry in forensic speech science: A problem to be taken seriously, is still available online.
Cornwall are not the first or only council to deploy VRA, there are others around the country doing so and the Internet is littered with Freedom of Information requests to councils asking them if they use it and how they apply it. Some such as Harrow have tried it, found it ineffective and abandoned it. The Department of Works and Pensions actually commissioned an evaluation trial that ran from 2007 to 2010 in twenty-four different regions. The final sentence of the final report from this reasonably thorough two phase trial was “No further trialling of VRA is planned and on the basis of this evaluation we cannot make any recommendations for its use within benefit processing.”
Just a few months afterwards this more positive but qualitative, evaluation was released by the DWP. It focussed more on how people felt using VRA and how fast staff could work through cases rather than on accuracy. Perhaps this was deemed more important as the system continues to be used to this day in some regions, perhaps one near you.
Ben Goldacre and others have recently argued for a greater use of trials and evidence to guide public spending. To those that agree with this approach the decision to outlay money in rolling out VRA must appear to be a very odd one indeed.
By Patrick Redmond
1 comment:
Ha, thanks for that, great link.
Post a Comment