Appalling as the Rotherham sexual abuse revelations have been, their awfulness and the immediate desire for heads to roll shouldn’t blind us to a scandal within a scandal. Sexual exploitation of children evokes a special kind of horror, but for purposes of prevention it is no different to domestic violence or any other kind of serious neglect. Like them it is preventable. The most important thing about Rotherham (or Rochdale, or Blackburn, or Oxford) is not the ethnicity and culture of the culprits, it is that it is in a direct line from Victoria Climbié, who was tortured to death in 2000, and Baby Peter Connolly in 2007 – a graphic illustration of the same management failures, multiplied 1,400 times.

At their heart is a Fordist concept of public services driven by fear, risk aversion and obsession with cost, all of which magnify the factors they are trying to control.

All post mortems of public service shortcomings find the same collective failure of understanding that serves to amplify demand rather than reduce it. Users are treated as separate incidents or episodes, leading to repeated assessments, referrals and opening and closing of cases without ever solving them. Families can be on a council’s books for years at direct and indirect costs of hundreds of thousands of pounds and end up no more stable, sometimes less so, than before. Because agencies work in separate silos, no one joins the dots, red lights are missed and opportunities for intervention are passed up.

For all social workers’ groaning caseload, few of the cases they see are actually new. Almost everything that comes across their desk is a manifestation of repeat or ‘failure’ demand (demand caused by a failure to do something or do something right the first time) – and it typically emanates from a small number of families, some of whose dysfunctions are registered by separate agencies but none of whom is ever seen in the round. Whatever the form the demand takes – mental or physical health problems, violence, truancy, drug and other abuse, antisocial behaviour, crime – that or other aspects of the chaotic family life of which it is a symptom will be known to one agency or another. In exploratory work on organised crime by Greater Manchester Police, officers were astonished to find that gang members, far from being undetectable masters of crime, were well known to other agencies, if not the police, and exhibited many of the characteristics of other dysfunctional families. Organised crime, said one police officer working on the project, was just one more symptom of out-of-control lives and incoherent responses to it by the community and public services. Sexual abuse is part of the same syndrome: as it now emerges, spouses and children of the abusers will likely have come to the notice of the police or council services – and have a high probability of suffering similar horrors if they aren’t rescued in time.

Why do these things keep happening? One problem, says Joanne Gibson, a senior consultant at consultancy Vanguard, which works across many public services including child protection, is the kneejerk reaction of politicians, media, policymakers and policy implementers to treat it as a people issue: the assumption being that risk can be reduced by controlling and monitoring what frontline social workers are doing. As a result the work is driven by process and bureaucracy designed to meet the hierarchy’s need to be accountable, not the needs of children or families.

‘Time and again when we study these systems end to end there is a catalogue of system, not people, failures,’ says Gibson. Social workers’ attempts to engage are hampered by an inflexible, form-driven process that prevents them from taking the time to understand the family context and history. Because the work is fragmented, no one gets underneath the surface narrative the family has invented to get into (or out of) and navigate the system – ‘no one really knows the child or family’.

This is compounded by a regime of thresholds (rationing) born of perceived financial pressures that has the perverse effect of keeping vulnerable people out of the system until abuse has happened, making rescue or a return to stability that much harder. The result is a game of pass-the-parcel with people who are already bouncing around the system, leading to the perception that demand is rising. But underlying demand is stable; what is going up is demand created by the system itself. As usual, attempts to manage by cost rather than purpose just push costs up.

By the time that families are officially ‘troubled’ it may, brutally, be too late to get them back; containment may be all that’s realistically left. As pioneering councils like Portsmouth, Stoke and Bromsgrove are discovering, the earliest possible intervention is essential to get lives back on track while it’s still doable – and that principally involves spending time to understand families in the round, in their context, not the producer’s. On the basis of work with an admittedly small number of needy families, Stoke is finding that a ‘rebalance me’ approach of prevention and understanding need in context reduces levels of dependency (and thus demand) across a spectrum of services. This isn’t a cost, it’s a human investment – and it has the side-effect of potentially saving millions.

Could this catch on? In 2011, Eileen Munro, a respected social work academic, published a report on child protection that laid bare many of the faults of the defensive, rule-bound current regime and recommended a shift away from targets and statutory guidance towards a child-centred approach that emphasised learning and local innovation. Frustratingly, says Gibson, although there is lip-service to the report at the centre, little has so far fed through to the front line where managers paralysed by fear of recrimination are even more reluctant to trust professional judgment, particularly when austerity measures are slashing staff levels and ramping up caseloads.

As in all public services, the issue is not one of resources but system design. ‘If there is to be accountability and something held to account, then it should be the system and its current design,’ says Gibson. ‘Blaming people publicly will just reinforce the risk aversion that inhibits people on the front line from doing the right thing. The result is a whole load of new bureaucracy and inspection that effectively locks down the service and by forcing skilled caring people to comply with it increases the risk of yet more young people losing their lives’.

It’s hard to conceive of it now, but in the late 1960s and 1970s, Whitehall and the public sector knew as much about IT, and in some areas more, than the private.

Then began a process which would later come to be popularised as outsourcing.

Roll forward 40 years, and here is John Manzoni, head of the government’s Major Projects Authority, describing the result. The wave of outsourcing that gathered pace in the 1990s, said the head of a body that monitors £500bn of public projects, has left Whitehall bereft of the ‘critical skills’ needed to procure and manage such projects. Execution and delivery were not ‘well-developed muscles’ in Whitehall, said Manzoni, who lamented that as a result of the outsourcing reflex, the government had lost crucial expertise in IT and technology and now lagged ‘five to eight years’ behind industry in these fields.

When outsourcing began to take off in the 1980s, it was sold as a simple win-win transaction. By focusing on what they did best everyone would benefit, as would the economy as a whole as the use of resources was optimised. ‘Outsource everything except your soul!’ exhorted the excitable Tom Peters.

From the outset, it was clear that it wasn’t quite as simple as that. For a start, outsourcers had to be not just a bit but massively more efficient for the arrangement to provide for both their profit margin and cost savings for the customer. Too often, contract terms gave no incentive for providers to improve. In the longer term, both the value of what was being given away and the hidden cost for the customer of not having an important process under its control turned out to be higher than expected.

Take the electronics industry. The Faustian nature of the outsourcing bargain was graphically revealed when Asian component makers eventually started eating not only the lunch but the entire body of western computer firms that discovered too late that in outsourcing ever larger chunks of manufacturing value they had inadvertently given away their soul too. Or aerospace, where obeying Wall Street’s strictures to minimise its asset base, Boeing outsourced so much of the manufacture of the Dreamliner, the 787, that the complexity of its supply chain outstripped its ability to manage it, causing delays to the launch and cancelled orders.

Or, to bring the story up to date, the UK public sector, whose travails, illustrating all the pitfalls described above, were the subject of Manzoni’s strictures. The e-borders fiasco, where the taxpayer has been landed with a £224m bill in damages and costs awarded to US defence firm Raytheon after a contract to upgrade UK border controls was improperly cancelled, is a timely reminder of the knock-on perils of dismal contract management. Calling the award a ‘catastrophic result’, Keith Vaz, chairman of the Commons home affairs committee, said in so many words that the UKBA didn’t have a clue what it wanted from the project.

The economist Joan Robinson once remarked that the point of learning economics wasn’t to acquire ready-made answers to economic questions, but to avoid being bamboozled by economists who put forward such things. The same is true for technology.

Consider the Department of Work and Pensions’ universal credit scheme, another prominent item on the MPA’s watch list. Curiously, UC doesn’t have a ‘traffic-light’ rating (green, amber-green, amber-red, red) in this year’s assessment. The reason given is that it is a ‘reset’. On examination, a reset turns out to be a variant on a dodge commonly used by service organisations to meet arbitrary targets and service-level obligations: closing a case which can’t be resolved within (say) the target time and then re-opening it as a new one, thus winding the clock back to zero. Opening and closing cases is a common ploy of IT help-desks and other outsourced services where the contractor is paid by volume. No prizes for guessing from which quarter the reset idea is most likely to have come, nor for thinking that those who dreamed it up will not have described it to government as what it is: a fiddle, a scam, a cheat.

Nor is it likely that consultancies whose livelihood depends on selling copious amounts of IT will suggest to government departments contemplating large-scale change that IT is the last, not the first place to start. IT is often glibly called an ‘enabler’ of change; but if done first, it is the reverse, locking in a design of work (and basis of payment) that are impossible to change subsequently except at enormous cost. Although less in the public eye than the central government projects, this is a growing issue all over the public sector. In local government, a number of shared-service and other outsourcing deals have descended into reciprocal recrimination when the promised benefits failed (as they do) to materialise because the starting design was wrong. In some cases councils have found themselves stuck with deals which they have learned the hard way to be disastrous, because they can’t afford to cancel them.

The prudent way of buying IT (and avoiding more cases like NPfIT, the aborted NHS computerisation programme, where large damages costs threaten to swell the £10bn already sunk in the failed scheme), is to put it last, not first. Computers are the servant not the master of change, which begins at the other end, redesigning a service to put humans upfront where, unlike computers, they can deal with the variety that human demand comes in. Such redesigns sometimes result in IT having to be removed as a constraint on doing things better; they are always less IT-intensive (and expensive) than before.

The moral of the story is that in order to outsource a process or service effectively, you need to know how to do it yourself, in every important particular. In which case, of course, you may often conclude that you’ll be doing yourself a financial and strategic favour by doing just that.

‘Like the population, the business sector of the US economy is ageing,’ says a research paper from the Brookings Institute, in an arresting phrase. It reports that firms aged 16 or older now account for 34 per cent of all US economic activity – up 50 per cent in 20 years. The share of all younger firms is correspondingly shrinking. As with jobs, housing and income for individuals, the business advantage is with the old and incumbent. With fewer startups (‘especially disturbing’), entrepreneurs are struggling across all sectors, according to the authors, with potentially unwelcome implications for productivity, innovation (where new firms have accounted for a disproportionate share of disruptive new product categories in the past) and employment.

The Brookings findings chime with other disquieting indications of sclerosis, not to mention mortality. The average life expectancy of firms is falling sharply, according to other research. This means that fewer are getting through the perilous pipeline of youth to become old and established. When they do, fading competitive vim means they have a greater chance of becoming entrenched and obese. If business is getting old and fat, ‘it appears to be getting fat because it is getting old – not the other way around,’ confirms the report. While the Brookings authors couldn’t find a direct link between ageing and consolidation, they did note that consolidation had increased over recent decades. In short, business is not only old and fat, it is also becoming more monopolistic.

Perhaps most striking of all, it is the publicly-quoted company, the central economic actor in the west, which is in steepest decline. In the US and UK, the most stock-market-oriented economies, listed corporations have been dying off like flies during the noughties. The universe of quoted US companies, at a paltry 9,500, is a whopping 50 per cent down from the all-time high in 1998. Although less in Asia, the fall is happening worldwide. As the costs of being public go up (regulation, activism, scrutiny) and the value goes down (companies nowadays rarely need outside investment), companies have been going private, not going public, or going bust, in droves. Mergers too have played a part. The quoted company, the engine of capitalism for the last 150 years, is beginning to look like an endangered species.

It wasn’t supposed to happen like this. Ironically a large part of the collapse of the corporation can be put down to the triumph of the cult of MSV, maximising shareholder value. In June, Harvard Business Review ran a special three-article ‘Spotlight’ section asking, ‘Are Investors Bad for Business?’ Two of the pieces, and a third indirectly, answered ‘Yes’. Basically, Wall Street (and especially hedge-funds and other ‘activists’) demands a high return on assets. One way to improve the ratio is to grow revenue and profits (the numerator) organically – but that’s hard and often slow work. Easier and quicker to slash assets (the denominator) by dematerialising, like Nike: outsourcing everything that moves while restricting investment to strictly efficiency-creating measures. Share buybacks, now being implemented in staggering quantities on both sides of the Atlantic, help increase leverage and force up total shareholder returns by the same mathematical tactic. The result is a weird kind of corporate anorexia: behind the apparent obesity, there’s nothing there. Corporations are auto-digesting. Under the impact of their perverse incentives, CEOs and short-term shareholders reap fabulous rewards while Innovation and job-creation rates are slumping. Meanwhile, the bulk of retail shareholders, and those reliant on companies for their retirement, are much worse off – the rates of return on assets and invested capital for US capital in 2011 were just one-quarter of what they were in 1965.

In this perspective, the monopolistic, or at least oligopolistic, tendencies at work in so many industries today – banks, retail, oil, automobiles, energy, phones, utilities, internet, to name a few – should be viewed alongside the Brookings findings as further symptoms of a damaged, unbalanced business ecology whose sustainability is now in serious question: in other words evidence of weakness rather than strength. In one compensating domain, however, giant old companies have undeniably increased their power: politics. With so much vested interest at stake, for monopolists the incentives, and returns, to political lobbying are sky high – which explains why CEOs think it’s more valuable to spend time schmoozing with government officials than selling to customers. Large single industry interlocutors suit governments too: but as the FT’s Tim Harford points out, such a cosy relationship is conducive neither to a healthy democracy nor a vibrant economy. A government in cahoots with Google and Facebook on surveillance doesn’t bear thinking about (although for the sake of prudence we should). From the point of view of the health of the economy as a whole, giving massive injections of cash to prolong the existence of wheezy overweight companies like the banks and General Motors now looks even more misguided than it did at the time.

After the south of England was hit by the hurricane of autumn 1987, the felling of 700 mostly mature trees in the famous collection at Kew Gardens was viewed as an irretrievable catastrophe. It has proved the reverse. The knowledge gained in the storm’s aftermath, say keepers at the arboretum, has revolutionised the science of tree planting and conservation, led to renewed plantations and given a vigorous second lease of life to some of the park’s most venerable growths. There’s an obvious lesson there for those whose job is tending the health of business, too.

Read my reflections on the Foundation forum’s dicussion on the innovator’s dilemma here

Sometimes an overnight revelation takes 20 years.

When I started writing a weekly management column for The Observer in 1993, I didn’t have an overall ‘theory’ about management. I knew it was important, that it was about people, and I sensed it was more craft than science. I knew it was about more than shareholder value, and suspected that, as in other fields, short cuts would turn out to be the reverse: organic growth was likely to take us to a better place than growth by acquisition and financial engineering. I wondered if I would run out of subjects.

There were also a number of puzzles, often to do with the glaring gap between reality and the rhetoric. Big companies were clearly essential to developed economies, but despite the comforting prose of their annual reports, why did they have to be such dispiriting places to work? Could tobacco companies and hamburger chains really achieve model citizenship through programmes of corporate social responsibility while their products were indirectly imposing huge costs on the rest of society? What about giant retailers whose ‘efficiencies’ (ie low prices) came at the expense of suppliers (sometimes whole industries) and low-paid employees?

More generally, if managers were the hard-nosed pragmatists and management the empirically-based discipline that convention says it is, why were they doing so many things that were at best ineffective and at worst harmful, even in their own terms? Research repeatedly said that acquisitions mostly destroyed value, but that didn’t stop M&A hitting record levels year by year. Companies that aggressively pursued shareholder value didn’t seem to do better than those that had a purpose other than maximising investor returns, at least in the long term. Intrinsic motivation was much praised when it applied to nurses and carers, the sense of vocation used to keep wages at subsistence level, so how come CEOs needed extravagant extrinsic incentives to persuade them to deliver a good day’s work? Especially since no one could find a link between CEO pay and company performance (not, it should be said, for lack of trying). Companies have spent most of the last two decades putting their supposed ‘greatest asset’ out of work, and the financial crisis and its aftermath revealed for all to see just what the financial sector really thought of the customer who is meant to be king. A major disappointment was the New Labour government that came to power in 1997, which instead of making the UK a role model for enlightened public-sector management simply grafted on to public services the industrialised practices that were turning customers off in droves in the private sector.

By the mid-Noughties it was hard not to believe that there was as much wrong with present-day management as right. Writing a weekly column was an extraordinary compressed education. On the one hand the work brought contact with leading management thinkers and experimenters, and on the other with readers who brought the fads and theories back to a touchstone: never mind the PR, here’s what it was really like to be managed in modern Britain. It was this combination that counselled caution in the face of the triumphalism that accompanied the fall of communism and later what was optimistically billed as ‘the Great Moderation’. Sceptism was of course vindicated in spades by the spectacular implosion of the financial system in 2008: we were right, management didn’t do what it said on the tin, and now, knitting together what I had sensed before, I thought I could see why, although I struggled to express it.

But although I had most of the pieces, the final epiphany only came later. It arrived in three parts. One was at a conference in Brussels last February, put on by an enterprising Czech-based NGO, the Frank Bold Society, on The Purpose of the Corporation. The briefing included a memo which set out the legal position in black and white: across jurisdictions, as a matter of law, shareholders don’t own the corporation, and directors’ fiduciary duty is to the company with which they have a contract. So in brief, shareholder capitalism, and the whole theory of corporate governance that has evolved to sustain it, including the assumptions about human nature and behaviour that it is supposed to control, is based on a myth.

The second ‘aha’ moment was at a Vanguard conference on health, some of the profound findings of which I wrote about here. One of them was that the thinking that would make the difference between a manageable and unmanageable NHS was not inherently difficult: it was just different. So different, in fact, that the existing management worldview couldn’t be modified to incorporate it – change could only come if that worldview was replaced. That helped to explain why initial resistance to the ideas was so strong.

The third element was an invitation to a workshop put on by the alumni of the Open University’s Systems Thinking in Practice course. The aim of the event was to give support and sustenance to systems thinkers who, for the reasons outlined above, could easily find themselves isolated and discouraged at work. I had expected to be interested and stimulated by the occasion, but it turned out to be rather more than that. Slightly unwillingly I found myself participating in an exercise designed to draw the lessons from a situation where systems thinking had helped in the past and consider how to apply them again in the future.

Bingo! Suddenly, reflecting on my trajectory, I could see what had been staring me in the face all along. It’s a system, stupid. The management apparatus that has been developed in business school and university economics and finance departments to further shareholder value and control is all of a piece, from governance, through the measures and techniques used, right down to performance management on the shop or call-centre floor. If the organising principle of shareholder primacy can’t be justified, it’s not an accident that so much of management designed around it is ‘wrong’ – the surprise would be if any of it were right.

Here’s the reason why management gets inexorably more complicated and regulation more intrusive, as ever tighter rules are drafted to deal with its increasingly harmful side-effects, whether for individuals, society or the planet. Management has become its opposite. It doesn’t solve problems, it creates them. And the companies it animates are a Frankenstein’s monster.

Everything you know about management is wrong. Literally.

‘Who cares about bloody management?’ asked the late Felix Dennis in exasperation. Many if not most people would doubtless nod in heartfelt agreement.

Yet there are powerful reasons why bloody management matters – and why you should care about it a lot more than you think.

First, we have to agree that management is necessary, a technology that makes possible things that we couldn’t do or want to do without – hospitals, transport, the World Cup, for example. In reality we don’t live in the atomised market economy beloved of the economists but an organisational economy in which large companies are the prime movers. Pace Adam Smith, our dinner arrives on the table not courtesy of the individual butcher, brewer and baker but of Tesco, Sainsbury and other management-intensive large firms, the marshalling yards of the economy, in Sumantra Ghoshal’s felicitous phrase. As the social technology of collective human achievement, management, as Peter Drucker put it, is society’s unseen central resource, and a modern developed economy is unthinkable without it. As he also remarked, it is what made the 20th century possible.

Purely physical technologies are neutral. A hammer, a wheel or a semiconductor isn’t good or bad in itself. It can be used for ends that are productive or non-productive, bad or good. In terms of use, management is similar, in that it can be employed alike for criminal or murderous intent – as in the Mafia or concentration camps – for blameless, life-enhancing ones such as charity or medicine.

But in another crucial sense management isn’t neutral at all. Management can be bad or good, in both senses of those words. Many technologies don’t come with a choice – a wheel is round, a hammer has a weight at one end, a semiconductor is the product of the physical properties of its materials. But the means management uses aren’t fixed. Unlike a wheel, management depends on its founding assumptions, and these can evolve. So can management styles. Today’s management isn’t god-given or inevitable, and nor is tomorrow’s. We can change it, and how we choose to do that matters profoundly. It will decide if management is a force for good or a force for bad.

As I suggested in my last article, management is the difference between an NHS (and other public services) that works and one that doesn’t. The financial crash of 2008 wasn’t caused by impersonal financial flows but by faulty management decisions made in boardrooms and offices on Wall Street and in the City of London. Ongoing scandals like Libor that still rumble around the international finance system like distant thunder were man- (or management) made in the same places. The way line managers manage is overwhelmingly responsible for the dismally low levels of employee engagement everywhere. According to the latest Gallup State of the Global Workforce survey, just 13 per cent of employees come to work each day emotionally invested in the organisation and focused on doing a good job, about half the proportion that is negative and potentially hostile. That leads the Drucker Society’s Richard Straub to suggest that what the world economy needs now is not more economic stimulus but a ‘Great Transformation’ of management that would not only add percentage points to world GDP but also reorient that growth towards sustainability. We could manage our way to prosperity.

Within Dennis’ expostulation, though, lies a powerful insight: it’s bad management that is bloody. Management only matters in use, as a means to an end. Management for management’s sake is meaningless. So the less of it the better. Good management is simple. Dennis mostly got it right, so he didn’t need to fret about it. In his influential Good to Great, Jim Collins found that companies on the way up paid ‘scant attention to managing change, motivating people or creating alignment’. They didn’t even worry much about pay. They didn’t need to, because their people were focused on doing a good job. This could be called management by getting out of the way – the opposite of the situation lamented by Drucker, where ‘so much of management consists of making it difficult for people to work’. Conversely management burgeons as managers struggle to do the wrong thing righter. For example, targets proliferate to get people to focus on parts of the job that other targets made them ignore. As Collins noted, ‘the purpose of bureaucracy is to compensate for incompetence and lack of discipline – a problem that goes away if you have the right people in the first place.’

In that sense, the crushing burden of today’s management is faithful testimony to its wrongness. But incompetence is not the worst danger that ‘bad’ management represents. Today’s management is ideological, not scientific, and running through it like the lettering in a stick of rock is a reductive, self-interested view of human nature in which people need to be bribed, whipped and constantly supervised to do their jobs. The danger is that these assumptions are self-fulfilling. People treated as untrustworthy do their best to escape supervision, thus justifying (in the negative view) more and tighter surveillance. Chief executives taught that they require incentives to perform are only too happy to believe it (greed is good!) and demand them in ever greater quantity. To put it brutally, today’s management moulds human nature after its own shrivelled, grasping image – a caricature of human potential.

The reverse is also true. Trust tends to breed trustworthy behaviour as well as greater engagement, and high-trust, high-engagement organisations by definition need fewer rules and regulations and less machinery of compliance – less management, in fact. That also means less cost. So ‘good’ management pays off twice: the first time by being more productive, and the second by reaffirming the positive, affirmative side of human nature. It is literally a force for good.

So sorry, Felix, this time you’re wrong. Everyone needs to care about bloody management. The future of the planet may depend on it.

How about a good news story about the NHS?

In 2008, the acute stroke unit at Plymouth’s Derriford Hospital was under as much pressure as its patients. It was bottom of the regional mortality table, the experience for both patients and relatives was poor, and every patient it treated was costing £2,000 more than it got back in payments. That added up to a deficit of more than £1m a year.

Yet while stroke was under scrutiny, there was no prospect of extra money for more beds. Counterintuitively, that was the best thing that could have happened. If money had been on the table, Plymouth neurologist and stroke specialist Steve Allder, the only person who believed that improvement was possible without spending any, would never have had the chance to put his ideas to the test.

Less than 12 months later, Allder was proved right and everyone else wrong. Plymouth now sat near the top of the mortality table. The patient experience was dramatically better. Quality was up. Yet far from increasing, the number of acute and rehab beds had fallen from 56 to 39. And instead of making a loss, the unit was now in surplus as the cost of (much superior) treatment was halved.

This transformation so flatly contradicts the universally accepted NHS narrative that less money equals worse service that it is hard to believe. Where could such a result have come from? The answer is simple: a different way of thinking.

This is the starkest illustration I know of something that politicians, and most other people, criminally ignore – the extraordinary leverage of management, for both good and ill. Viewed through the lens of conventional management, the NHS, and public services generally, are doomed to penny-pinching mediocrity as ever-increasing demand hits finite or diminishing resources. But under another lens a very different reality comes to light. Underlying demand is not going up – indeed, it is both predictable and stable. And while there is indeed a resource crisis, it is about the way resources are deployed, not the amount. There is scope for improvement; there is hope!

When Allder studied Plymouth stroke patterns, he found that Derriford predictably and regularly admitted 1.5 stroke sufferers a day. Much more variable was patient length of stay, which when analysed revealed that by far the greatest resource – 75 per cent of bed days – was absorbed by a small number of already frail people, suffering from massive strokes from which they were unlikely to recover. They (and their relatives) were also least well served by the existing system, often kept tenuously alive only to prolong agony and distress. The answer was to redesign the pathway for this group to accord with their and their families’ wishes, and to be more proactive with other groups – systematically ensuring that those who could benefit from rehab got it, for example. The result: the blockages caused by the long-stay group were resorbed, more patients could be seen in timely fashion, and the perceived need for extra beds went away.

Looking at the rest of the hospital, Allder found exactly the same patterns for other conditions: stroke was a microcosm of the whole. The consultancy Vanguard, which has been working in the NHS for the last three years, would go further and say that the same things are true of the NHS as a whole. That is, overall demand over the system is stable, a disproportionate amount of resource is absorbed by a small proportion of intensive users, and understanding the pattern of demand is the key to the Rubik’s cube of reconfiguring resources for permanently better and cheaper results.

Why is this evident in one optic yet invisible in the other? Back to Allder. He explains that the current stroke regime has more than 120 quality measures; when problems arise, they are addressed successively, in isolation. ‘After you’ve done six, there’s no money left, everyone’s exhausted and nothing has changed.’ At a more abstract level, conventional managers view a hospital as an asset to be sweated like any other kind of plant. They aim for efficiency through economies of scale, leading them to develop functional designs that use cheaper resources for simpler activities which they then run for high utilisation and low unit costs.

Unfortunately, as W. Edwards Deming taught many years ago, you can’t optimise the whole by optimising the parts. The first complication, notes Vanguard’s health lead Andy Brogan, is that fragmenting the world and running it for productivity makes for complicated governance and accounting across functional jurisdictions as each seeks to optimise and defend its own domain.

Second, it assumes that lower unit costs (making an appointment, seeing a nurse, doing an assessment) equates to lower overall costs. This is a fundamental error. It blinds managers to the fact that not all demand is equal. Much of it consists of what Vanguard terms ‘failure demand’, that is knock-on demand created by the failure to do something or do something right the first time – repeated appointments, referrals and assessments, people shuttling between A&E and GP surgeries to get their problems fixed. Failure demand accounts for all the apparent demand increase in health and many other public services, and often an enormous proportion of the total – up to 80 per cent in the NHS, Vanguard estimates.

Failure demand explains why lower unit costs don’t result in lower overall costs. Standardised services can’t meet the variety of need, and however cheap are a false economy – they end up increasing cost. Failure demands explains why organisations can be simultaneously frantically busy and highly ineffective. As Brogan puts it: ‘We can solve the wrong problems until we’re blue in the face, or we can get under the skin of what demand is, how we can create value for people, and what expertise we need to do it’.

Pause to consider the implications here. The measures that managers use to run critical environments like hospitals not only do not tell them the things that really matter, like quality and value to the patient, they actively mislead. So they don’t see that the effect of the focus on efficiency and productivity is to create more failure demand faster. In other words, not only are they not improving things, across the health system they are making them worse. Here is the explanation of the damning finding that with all the money spent on it NHS productivity, like that of the public sector in general, is actually going down. We have designed organisations that not only don’t know how bad they are, they don’t know that they don’t know it.

So what is the lens that makes this dynamic visible and in so doing opens up the organisation to change? In this case it is an approach to service organisation developed and honed by Vanguard over the last over 30 years that treats the organisation not as a collection of parts but as a whole system.

For practical purposes it starts with a definition of purpose from the point of view of the citizen or user – for stroke victims, timely and accurate diagnosis and treatment, safe recovery and earliest return home. It then asks how well the system meets the purpose (usually, as with the stroke unit, no one knows, because the measures in use have no relation to purpose). Then, having deconstructed the work to see what contributes to purpose and what doesn’t (usually a lot), it reconfigures resources predictably and reliably to meet the demand. As the organisation gets more closely aligned with purpose, costs drop out of the system in the form of less wasted effort and above all reduced failure demand – although crucially how much is impossible to specify in advance because cost-reduction is a side-effect of doing something better, not an end in itself.

While none of this is hard to comprehend, the challenge is putting it into practice. Although redesigning a medical pathway is in itself non-trivial, the issues are not primarily technical. One factor is the cross-functional governance issue described by Brogan. More challenging yet, the cheerful landscape projected through the new lens is so unfamiliar and disconcerting that ironically people are frequently more sceptical of it than of the familiar topography of failure.

Precisely because the approach is conceptually simple, it’s easy for senior managers used to dealing in top-down abstractions not to ‘get’ its full import. Understanding that there is a dynamic between purpose, measures and demand is one thing; realising that accepting it means rejecting all previous management certainties (economies of scale, standardisation, targets, conventional performance management, outsourcing…) is quite another. As Brogan notes, a niche for it can’t be found in the existing management worldview – it replaces it with a different one. Allder acknowledges: ‘At the start I thought I was doing a bit of improvement. As it went on I realised that it was an entirely different strategy.’

This is not a fad. In effect, a new management paradigm is struggling to be born. The infant is undeniably vulnerable to changing circumstances and personnel – Allder recounts that after the last election, promising initiatives at Plymouth were cut short when money became available for easier management options, while managers appointed to ‘get a grip’ often do so literally, reverting to default postures of top-down control and short-term cost-cutting that are incompatible with the new approach.

On the other hand the personal ratchet goes one way: for those who get it, there is no going back. This is why eventually a tipping point will come. The old ways are no longer affordable and, if painfully slowly, confidence that there is a better alternative is growing. ‘It’s a call to action,’ says Allder. ‘The potential for improvement using these methods is fantastic. Across conditions 40 per cent of our cash goes on long-stay emergency in-patients. I have no doubt that adopting a similar study for each condition we could do it 50 per cent better and cheaper.’

Read my article ‘Well done – you’re fired’ for the Foundation here

‘Prison governors have been ordered to cut the cost of holding inmates in England’s bulging jails by £149m a year, as part of a radical programme designed to slash the costs of incarceration by £2,200 a year per prison place’, reported The Guardian recently. The ‘savings’ were to be found through economies of scale (closing smaller prisons and building bigger ones), replacing prison staff with CTV cameras, and an enforced ‘benchmarking exercise’ designed to drive prisoner costs in higher-cost jails down to the level of the lowest. Meanwhile, the justice secretary has responded to two cases of high-profile prisoners absconding from open jails by tightening the licensing and temporary release rules.

Taken together, these are among the most depressing pieces of news in the last six months. They are a veritable compendium of stock management responses and assumptions (size, cost, technology, control), all of which are misleading, counterproductive or wrong. They’re back to front, stating the answers before the question. It’s management on auto-pilot, without thinking or learning, a continuation of the policies that have seen the prison population double to 85,000 since the 1990s, thus causing the cost explosion that the current initiative is supposed to mitigate, only on steroids.

These are the same outdated industrial strategies, born in the mass-production factories of the mid-20th century, that continue to wreak havoc across the NHS and the rest of the public sector. The benefits of scale are always overestimated, partly because the wrong things are being measured, and the diseconomies ignored, for the same reason; managing cost always raises cost because it starts from the wrong end – lower costs are the earned consequence of redesigning the system to work better; technology is invariably used to do things that should be done by humans and vice versa; tighter controls focus on doing the wrong thing better at the expense of doing the right thing. It’s a locked-and-bolted certainty is that as with A&E units or local authority benefits offices, the remedy will make things worse and more expensive.

Starting with cost per prisoner year is a classic case of management being led astray by an irrelevant measure – one that calculates activity cost but gives no indication of how well the system is performing overall against its purpose. Only measures related to purpose are any use to anyone, whether politicians or managers, trying to figure out how to do things better.

Let’s assume for argument’s sake that the purpose of prison is to protect communities and enable the maximum number of people to live stable lives. In that case, even a cost per prisoner year of zero would be a false economy if achieving it means putting offenders in larger prisons which have higher reoffending, suicide and self-harm rates than smaller local establishments where the offender can stay in touch with their family, friends and local services and thus stands a better chance of reintegration into the community when they come out. Larger prisons, such as Oakwood, the G4S-run jail that sets the cost benchmark, are often drug-ridden, impersonal, violent and hard to control, making them perfect academies for criminals and breeding-grounds for addictions and dependencies that pass on massive costs to other public agencies.

Consider the prison population (‘demand’) as a three-tiered triangle of which the apex comprises a small minority of unreconstructed serial offenders who absorb the bulk of the prison resource. At the bottom (the biggest section) are those who are in prison through bad luck, temporary misfortune or because their lives have fallen into chaos. In the middle are prison’s ‘floating voters’, offenders who could go either way: either joining the ranks of the hard cases or with a bit of help getting their lives back on track so that they can rejoin society without endangering themselves or their communities.

Since between 50,000 and 70,000 offenders rejoin society every year (it’s self-evidently too expensive to keep them in for ever), the effects of being locked up really matter. In all three categories, the only conclusion is that traditional prison does a terrible job – not just wasting resources, actually exacerbating the problems it was meant to solve. Recidivism statistics suggest that prison creates more violence than it prevents. Hardened criminals get harder and more prolific (hello, Skullcracker). Prison is poor at deterring or rehabilitating – nearly half of those who leave are back behind bars within 12 months, more in the case of those on shorter sentences, and, shockingly, three-quarters in the case of young offenders. As for the lowest-level demand, much of it occurs because, as with A&E departments, prisons are a plughole of last resort, the recipient of all the miscellaneous problem demand which has nowhere else to go. Most offenders in this category suffer from problems that prisons aren’t designed or equipped to solve – very few don’t suffer from one of mental, learning or other disability, drug or alcohol dependency, homelessness or unemployment or a combination of the above. They shouldn’t be there at all.

From all this it’s fairly obvious that the current prison cost-cutting exercise is an irrelevant response to the wrong problem. Crime prevention and treatment cries aloud for the kind of holistic, locality-based approaches to social policy issues that are being tried out with promising results by councils such as Stoke, Bromsgrove and Camden, along with many other agencies in the public sector. In the US, in the rare places where such therapeutic initiatives have been applied, they predictably end up saving many times their cost, by reducing overall demand. The best rehabilitation programme? Prison degree courses, which have been ‘shown to be 100 per cent effective for years or decades at a time in preventing recidivism’. Gardening also shows promising results. As for prisons, it’s amazing that we tolerate the continued existence of institutions with such a grotesque and expensive failure rate – literally throwing good money after bad. As a US professor of psychiatry puts it, in time ‘prisons will come to be seen as a well-meaning experiment that failed, rather like the use of leeches in medicine.’ But no politician would dare to say that.

It’s simple. There is only one question that MPs needed to ask Pfizer’s Ian Read, or any other CEO pursuing a takeover of national size and scope: to what or to whom is your primary obligation as CEO?

Read would have probably said something like ‘building shareholder value by delivering innovative and life-improving therapies’. In its 2013 annual review, Pfizer states its mission as ‘to be the premier innovative biopharmaceutical company’, its four strategic imperatives being to ‘innovate and lead’, ‘maximise value,’ ‘earn greater respect’, and ‘own its own culture’.

The honest answer, however, is ‘maximising value for shareholders’. That is what most directors (including AZ’s) believe and governance codes suggest. And it’s certainly the story that Pfizer’s figures tell. In his letter to investors, Read boasts of handing them nearly $23bn in share buybacks and dividends in 2013, bringing their total cash returns over the last three years to $53bn. Not worth a mention is that that was more than double Pfizer’s R&D budget of $22.8bn and comfortably exceeded its entire net profit for the period – an omission that is as eloquent about the company’s priorities as the numbers.

If shareholders rule, by definition commitments managers give to anyone else aren’t worth a packet of aspirin. But this is just one of the toxic consequences of shareholder primacy that the Pfizer episode points up. Shareholder primacy combined with a fierce market for corporate control, another essential element in Anglo-US governance, results in ‘institutionalised short-termism’, as John Plender put it in the FT, which favours buying and selling over the harder work of boosting shareholder returns through organic growth. ‘Big pharma is no longer in the innovation business, using its own resources to fund the high-risk ideas,’ says Sussex University’s Mariana Mazzucato. It prefers to concentrate on near-term development and rely on small biotech labs and publicly-funded academic research to do the front-edge work. Cutting research overheads and taking advantage of the UK’s lenient tax rates to benefit shareholders (executives prominent among them) is transparently what the AstraZeneca deal was about. This is tantamount to socialising risk and privatising reward, Mazzucato argues – nor much different from the banks.

There are three mighty ironies here. In the aggregate, investors in whose name executives claim to act do not benefit from a shareholder-first regime. Former dean of Toronto’s Rotman School of Management Roger Martin has calculated that shareholder returns from 1977, the beginning of the shareholder value era, to 2010 were lower than in the period from 1933 to 1976 when managers were supposedly empire-building to feather their own nests. The only ones who do consistently gain are short-term opportunistic shareholders like hedge funds – and top executives. Here again, Pfizer is an exemplar.

Its current market capitalisation of $185bn is considerably less than the $240bn it has spent in the last 15 years on three large acquisitions. So it has destroyed shareholder value. But it’s not executives who pay the bill. In 2013 the six top managers listed in Pfizer’s proxy statement to the SEC took home $52m between them, more than half in stock. Their average salary of $8.6m is 112 times the Pfizer average and 309 times that of the company’s lowest-paid workers. This is another consequence of shareholder first: the perverse management incentives it generates are the engine of inequality, the wedge pushing top salaries ever up and wages at the bottom ever down. Here is a perfect illustration of the rise of the Thomas Piketty’s corporate ‘super-managers’, at the direct expense of their underlings’ pay and pensions.

The second irony in shareholder primacy is that as well as not working very well it is factually wrong: a fraud based on a myth. ‘Corporate law does not, and never has, required directors of public corporations to maximise shareholder value,’ writes Lynn Stout, distinguished professor of corporate law at Cornell University, in her critique, The Myth of Shareholder Value. ‘Second, closer inspection of the economic structure of public corporations reveals that shareholders are neither owners, nor principals, nor residual claimants. Third, the empirical evidence does not provide clear support for the proposition that shareholder primacy rules produce superior results. Indeed, once we shift our focus from the performance of individual firms to the performance of the corporate sector as a whole, it suggests the opposite’.

In explanation, Stout notes that the theory came out of business schools and economics departments in the 1970s which had little regard for legal niceties; she attributes its enthusiastic uptake to its undoubted appeal to the two powerful interest groups that have monopolised the benefits, corporate managers and short-term shareholders, as well as to free-market ideologues. It has now congealed into an article of faith as impervious to questioning or doubt as any tenet of fundamentalist religion.

The third irony, of course, is that AZ uses exactly the same unreconstructed shareholder discourse as Pfizer, which is why it was vulnerable in the first place.

The implications for any such monstrous tie-up should be clear. A shareholder-value-driven entity of this size of Pfizer and AZ is too big and powerful not to have huge public-interest implications – and the public interest would be to ban it on those grounds alone. It would be too big to fail, and have every incentive to exploit that moral hazard for its shareholders. As the FT’s Martin Wolf decisively noted, the fate of such mergers should not be decided by shareholders, many of whom (as in the Cadbury-Kraft case) have no long-term interest in the company, but by the board in the interests of the company as a whole. Never trust a manager who is driven by shareholder value.