If the Baby P affair taught anything, it was surely that putting computers in charge of human affairs is wrong in principle and disastrous in practice.
Anyone doubting this should watch the half-hour BBC Panorama programme on the computer-led fitness-for-work assessments administered by Atos Healthcare under the government’s programme to cut the number of people claiming disability benefit and get more of them back to work. And it is now confirmed by the National Audit Office (NAO), which says that the contract has been poorly managed by the Department for Work and Pensions (DWP) and is not delivering value for money.
Principle first. Making people subject to decision by computer is demeaning, reductive and by definition inhuman. Would it be acceptable for a computer to decide the death penalty? I thought not. There’s a good reason why in every democratic state justice is dispensed by magistrates or in serious cases a jury of 12 human beings. Justice is a matter of judgment, of weighing different considerations against each other. It can’t be done by an algorithm. The independent reviewer of the fitness-for-work programme, Professor Malcolm Harrington, says that for many people the experience of undergoing the disability test is ‘traumatic’. His subsequent comment – ‘I think people are being treated more like human beings now, but it is still difficult to go through it’ – is not a vote of confidence.
The principle links closely with the practice argument. To make a problem treatable by computer, it has to be reduced to numbers. But Deming’s dictum that the most important things in management are unknown and unknowable holds for the human condition too. A computer able to register and and put a weighting on all the variables would have to be as powerful and sensitive as the human brain – and we have those already.
You’d want computers to play a big part in running nuclear power plants or flying today’s passenger aeroplanes. NASA’s software for putting a man on the moon was flawless – an undeniably impressive achievement. But human beings are not reducible to equations in the manner of the physical sciences. Computers are good at the routine and predictable and bad at variety. Humans are the reverse. It is baffling, illogical and the worst of both worlds to use each to do what it is least good at.
Computers are flummoxed by variety; humans take it in their stride. The Panorama programme showed what happens when you do the reverse. While the Department for Work and Pension itself estimates that fewer than 0.5 per cent of claims for disability benefit are fraudulent, Atos Healthcare’s software (it’s a red herring that Atos is French, by the way – any other consultancy system would be just as bad) currently ranks 30 per cent of those called up for reassessment as fit for work.
These include Christopher Davies, an emphysema sufferer who can’t climb stairs or walk 50 metres without having to stop for breath; wheelchair-bound Shannon Thompson, who has bone disease and is permanently on morphine; and Steven Hills, who died of heart failure 39 days after being told he was fit for work for the second time. His first assessment, against which he had appealed, had also found him fit even though the assessor was so concerned about his heart that she had told him to see a doctor as soon as possible. You couldn’t, in other words, make it up.
Any human with a grain of sense could see that sending such people to work is absurd (although many of them in fact would love to) – what would they do? Overall 40 per cent of assessments (in some areas apparently nearly all) are revised on appeal – so why do they happen? Although the minister, Chris Grayling, denied that targets were to blame, an assessor told of pressure to conform to the ‘averages’; to classify too many as disabled was to invite attention from her manager. It is difficult to judge why so many of the decisions are wrong, according to NAOs head Amayas Morse, ‘as the department does not routinely request feedback on the rationale for tribunal decisions’. He criticised DWP for failing to seek financial redress for Atos’ mistakes and not demanding higher quality standards in the tests.
No one who has spent a minute thinking about systems will be surprised at the costs of this mayhem. First, there is the uncountable human cost. Did the stress of two Kafkaesque assessments contribute to Hills’ death? We can’t know. What we do know, because doctors told Panorama, was that, as in other areas, when a social service sets out to cut costs, it simply passes them on to the NHS which as last resort is forced to pick up the bill as distraught people besiege surgeries and hospitals for help with their unsolved problems. Then there is the cost of the Atos consultancy assignment – currently £112m a year for 78,000 assessments. Finally, factor in the cost of rework – appeals and reassessments – which is currently running at £60m. In effect, says the NAO, the tests are being paid for twice. When all these are taken into account, it seems likely that savings from this grotesquely applied measure, if any, will be minimal.
The obvious fact is that, unless the DWP’s fraud figure is a vast underestimate, removing disability benefit solves no problems and simply moves the cost somewhere else. As pioneering work with benefits of various kinds in Devon, Stoke and Somerset (to name but a few) show, the only durable way of cutting the cost of benefits provision is to solve people’s problems, whether employment, lodging, or need for any other benefit, as quickly as possible. This appears to be more expensive than dealing out impersonal, mass-commissioned aid packages in the short term, but in the longer term it’s cheaper because, in a mirror-image of Atos’ brainless and terrifying assessment system, it makes the problems that cause the cost disappear, so the sufferers don’t have a reason to turn up for help again.
What is the role of public services in the 21st century? Providing individual help that enables people to lead their lives as independently as possible. People can do that, humanely and surprisingly cost effectively. Computers can’t, and never will.