Big Ben is the emblem of London, and by extension of the UK too, in both sound and sight as representative of Britishness and British history as the Eiffel Tower of France or the Empire State Building of of the US. Yet this national icon is now in such disrepair that according to a parliamentary report repairs worth £40m are urgently needed to prevent the clock’s hands dropping off, the bearings seizing up and the pendulum slowing to a stop. As it is, London will be without its visual and aural signature for four months while repairs are carried out, or 12 if the clock gives up the ghost first.

It’s tempting to see in the state of the emblematic clock tower a metaphor for the UK economy as a whole – the stately exterior concealing the engine of a clapped-out 1960s Mini. Perhaps only in the UK is such vicious short-termism, a lethal blend of ignorance and expediency, conceivable.

It is not just Big Ben and the Houses of Parliament (which together, note, face a repair bill of no less than £7bn to stop them crumbling around MPs as they speak). The same effortless ability to fudge the inevitable and then be astonished at the outcome – a 156-year-old clock needs maintenance! whoever knew? – is visible everywhere. There’s the competitive boasting of politicians about their ‘business friendliness’ while the Bank of England warns that a financial sector geared to serving (and remunerating) itself rather than the wider economy needs to be reshaped to suit the needs of those outside rather than inside the charmed circle. The orgy of self-congratulation greeting the success of the James Bond franchise at the global box office glosses over the fact that the car marque inseparable from the brand, Aston Martin, is another British icon with bits falling off, swerving from crisis to crisis under the ownership of a foreign hedge fund.

Or take the steel industry. The fact is that UK steel has been in crisis since the 1980s, by which time it was already crystal clear that there could be no long-term future for steel firms in isolation making commodity products in competition with emerging low-wage economies such as (then) Korea or efficient high-tech ones like Japan and Germany. In 2014 Japan was still the second largest steel producer in the world and Germany the seventh, turning out respectively nearly 10 and four times as much as the UK, which has meanwhile tumbled from fourth world producer in the late 1960s to 18 today, its output having halved.

It’s also been crystal clear that the long boom in commodities, including steel, that has been fuelled by outsize Chinese growth rates would one day come to an end – and that day would be far too late for politicians to consider setting up task forces and other futile talking shops. The cat was let out of the bag by a steel industry insider on the Today programme who explained that the reason blast furnaces were firing full tilt in Japan and Germany while ours were going dark and cold is that in less short-termist and neo-liberal economies with more thoughtful management strategies, steel plants exist as part of vertically-integrated supply chains whose participants are tied together by long-term contracts which give them a powerful incentive to look after the interests of the supply chain as a whole as carefully as they do their own, the two being effectively the same. In other words, while competition is important, so is cooperation; an adversarial supply chain, in which each participant tries to cheat and bully its suppliers and customers, is a recipe for bankruptcy, and, and… now what else is that a recipe for?

Ah yes – horsemeat! Here’s part of what Manchester University’s Centre for Research on Socio-Cultural Change (CRESC) said about the horsemeat scandal two years ago: ‘There are better ways to organise the supply chain through vertical integration to ensure participants take responsibility for the overall health of the supply chain… The better way, which delivers on broader economic and social objectives, is represented by the integrated national models of the Danish and Dutch pig industry or the directly-owned processing of Morrisons… which aligns the interests of firm, supply chain and society’ – and kept the retailer immune from the contamination that affected other supermarket chains.

It’s perfectly true that British steel plants suffer from higher energy costs than many of their rivals. But that too is the result of similarly myopic, unsystemic thinking about power generation. Only the UK, an island nation floating on oil, surrounded by wind and waves pounding its 7,000 miles of coastline 365 days a year, could find itself in its present position of commissioning back-up power plants using diesel generators and buying vastly expensive nuclear expertise from nationalised energy industries in France and China to keep the lights from going out.

To complete the joke, all this is happening in a country where (hat tip to Douglas Board for this), more undergraduates and higher-degree students study business and administration than any other subject, and by some margin; and one of whose more successful industries consists of advisory and professional-services firms that make money from telling others to do as they say, not as we do – in other words, to use the technical term, from bullshit. Now, could one of these madrassas of capitalism kindly furnish us with someone who knows how to wind a clock?

Volkswagen is not alone or even the worst. Wittingly or not, almost all companies are built on lies and and self-deception, from top to bottom. Cheating is endemic.

Whether by financial or (in the VW case) physical engineering, executives routinely manipulate or ‘smoothe’ corporate results to meet stock-market earning expectations and trigger their own cash or stock-option bonuses. Recent research suggests that selection panels recruit top accountants who are willing to do just that over those who aren’t.

‘If you aren’t lying you aren’t trying!’ exhorted a Barclays manager quoted in the LIBOR-fixing enquiry. Lower down the organisation, people equally routinely cheat in small or large ways to make sales or other targets and keep managers off their backs. But small-scale cheating can all too easily morph into institutional self-deception and eventual scandal (the banks, Mid-Staffs, VW) in which, regular as clockwork, leaders protest their innocence and blame ‘a few bad apples’ for the misdeeds. At the extreme, as I’ve noted before, self-deception and denial can turn whole organisations into reversed-out versions of themselves: banks that make people poor, hospitals that make them medically dependent, shareholder-value-maximising companies that destroy value.

In view of the dire effects, why are lying and self-deception so prevalent? It’s not that companies are full of wicked people. It’s the system, stupid – as the telling phrase ‘gaming the system’ clearly announces. And the culprit? In a word, measures and the counterproductive ways in which managers use them.

Measuring is one of the most important things managers do. Without appropriate measures, management can’t be even methodical, let alone scientific. Yet given the lip-service paid them (how many times have you heard the expression, ‘What gets measured gets managed?), astonishing quantities of the measures that companies compile are useless or worse. While good measures promote learning, improvement and innovation, bad measures are often worse than no measures at all: not only do they drive actions that alienate customers and their own employees (yes, you do get what you measure), they keep managers in the dark about their effects.

The reason is simple and profound – but not obvious. As John Seddon has put it, whether organisations realise it or not, ‘there is a systemic relationship between purpose (what we are here to do), measures (how we know how we are doing) and method (how we do it)’. Measures should be a means to an end (learning, improving). When purpose is put first – doing what matters to the customer through products or service – and measures are related to it, managers and employees can see directly how well they are achieving that purpose. Their job is then, guided by the measures, to identify methods to do it more effectively. That is, the purpose-measures-method trio form a learning system. The job of measures is to connect actions with consequences.

Now look what happens when the purpose is not related to customers, or there is no explicit purpose at all. To fill the vacuum, the measure becomes the de facto purpose – the means turns into the end. But putting the measure first (as in sales or other targets, standards and specifications mandated by governments and regulators) has terrible consequences, because methods are now geared to achieving the measure rather than the real purpose (often at the expense of the real purpose). They are not about learning but making people accountable. And as all such targets are arbitrary, with no necessary relation to the system’s capability to meet them, there are high incentives for people to cheat. As W.E. Deming succinctly put it half a century ago: ‘People with targets and jobs dependent upon meeting them will probably meet the targets – even if they have to destroy the enterprise to do it’.

Cue Volkswagen, seemingly a classic case of capture by its own closed, self-referencing metrics. Fifty per cent owned by the Porsche family and 20 per cent by the state of Saxony, VW, as the company at the centre the company town of Wolfsburg, was focused entirely on volume and jobs, to which environmental protection and even the customer came a distant third. Everyone seems to have bought into those goals, including the supervisory board, the unions and shareholders who were doing very well from the formula. But volume meant selling in the US, which has much stricter diesel emission rules than Europe. With feeding the machine the priority, and detached from customers, it’s easy to see how self-confident, production-minded managers could convince themselves that cutting a few testing corners wasn’t a serious matter – after all, fudged emission tests don’t make them poor cars, just not quite what the ads said they were (ads also lie, as consumers well know; witness the soaring use of online adblockers). Little lies become bigger ones, until it is very hard to go back.

As set out in Goodhart’s Law – ‘Any statistical regularity will tend to collapse once pressure is placed upon it for control purposes’; or in plain words, a measure that becomes a target is no longer reliable as a measure – all data collected to meet incentives is worthless because liable to manipulation. The mother of all numerical targets, of course, is shareholder value, which is useless as a guide to action in the real world of customers and service, because it is unrelated to what matters to customers, is used as a carrot or stick rather than an aid to learning, and does not connect actions to consequences. It is owned by the boardroom, not those doing the work. Incidentally, it may well be true that departed CEO Martin Winterkorn did not have direct knowledge of the testing scam: in a delicious variant on Catch-22, in an organisation that uses measures to make people accountable rather than to learn, no one is ever accountable because they always meet their targets. Winterkorn’s deeper responsibility, of course, is for the system that drove such perverse behaviours in the first place.

As VW’s subsequent struggles show, the distorting effects of turning measures into ends are catastrophic. The company has set aside $6n to provide for vehicle recalls and rectifications, while liabilities for fines and possible lawsuits could dwarf that. Some executives may face criminal charges. If, back in the real world, car-buyers give up on the brand, things could get even worse. Spending is already being reined back. All of which shows that actions do eventually have consequences, even if the failure of official measures to make the link apparent means that they turn up many years later as an unexpected and very nasty surprise. Rarely have the wings of chickens flapping home to roost cast a longer or deeper shadow.

Harvard Business Review has published a slightly re-edited version of my post for the Global Peter Drucker Forum 2015, entitled ‘Staying human in the robot age’, here

Read my blog ‘Staying alive’ for Drucker Society Europe here

To say that Paul Mason’s PostCapitalism: A Guide to Our Future is controversial is an understatement (see a selection of reviews here). Even in the same newspaper, while one reviewer admires the author as a worthy successor to Marx (in a limited sense, admittedly), another dismisses him as a Trot ‘shackled to the remnants of a hopelessly impractical ideology’ that even Jeremy Corbyn would be embarrassed to embrace.

But there is an intellectual energy to this fascinating, challenging, puzzling, absurdly ambitious and sometimes equally annoying book that makes it an enthralling read even when you want to throw it across the room or can’t get your head round what he is trying to say.

As economics editor of Channel 4 News, Mason has made his name with vivid close-to-the ground reporting from Greece, Tunisia and other trouble-spots that stands out precisely because it seeks to locate immediate events in a wider, global context – as conveyed in the title of his previous book, Why It’s Kicking Off Everywhere. The TV reports are a microcosm, or perhaps a blueprint, for the wildy more grandiose project of PostCapitalism, which is to draw together things and events usually treated as discrete (Greece, the Arab Spring, austerity, migration, the long cycles of innovation and technological advance, successive corporate scandals culminating in the crash of 2008, the decline of the working class, automation and inequality, to name a few) into a single narrative thread describing the trajectory of capitalism. Not content with tracing the history and causes its internal crisis, greatly exacerbated by exogenous shocks like far-reaching climate and democratic change, Mason also seeks to project into the future a superior world beyond capitalism, where a lot of stuff is free and people are too, in the sense that they don’t work, or don’t work much, anyway.

As this suggests, one of the refreshing things about this book is its contrarian-ness – even its optimism, which is based not on resisting automation and the encroachment of computers into working life, as you might expect, but on embracing it, as fast and and far as possible. This is definitely not an old-fashioned Marxist scenario (at various points in the book Mason ticks off Marx, Engels, Lenin and a gallery of early Russian economists, and tips a hat to bogeymen of the left such as Friedrich Hayek and Ludwig von Mises). Nor indeed does he see it as inevitable. We have to want it and make it happen, too.

Briefly (it can only be briefly), Mason’s thesis is that ‘info-tech’, as he calls it, is different from anything that has gone before. It affects every actor in the economy, down to individuals. It is increasingly making work redundant – think of the Oxford Martin School prediction that almost 50 per cent of US jobs are vulnerable to automation in the coming decades, a figure which some now think an underestimate. But info-tech also does funny things to the price mechanism. In a world where the marginal cost of replicating algorithms, MP3s, MS Office or news stories is effectively zero, high prices and profits can only be maintained by monopolies (Apple’s walled garden, Microsoft’s lock-in, Google and Facebook’s appropriation of externalities in the shape of personal information involuntarily given away) – in other words by making artificially scarce what could economically perfectly well be abundant.

But if the inherent manufacturing cost is tending to zero, what happens, finally, when people deliberately decide to de-monopolise (sorry) products or services by collaborating in ways that ‘no longer correspond to the dictates of the market or the managerial hierarchy’ – for example, by jointly writing the free Open Source software that powers most of the internet, the most powerful computer in the world and the dominant mobile phone operating system (Android), or contributing to the largest information product, Wikipedia? The answer is that these non-market initiatives create something new – what might be called a brand-new ‘monopoly of free’. By removing whole chunks of enterprise from the market economy (as a commercial site Wikipedia could be taking in $3bn a year, by one reckoning), they effectively bar the terrain to the market. A new economy of abundance is in the process of being born.

Bring it on, says Mason. Capitalism as we know it, having spawned economic and technological forces that it can no longer contain, has run out of steam. It has always driven forward by opening up new markets, whether demographic, geographic or technological: but the monopoly of free makes the last frontier, the personal, harder to cross. Without it, capitalism is ‘an economic system in disequilibrium with its environment and insufficient to satisfy the needs of a rapidly changing humanity’.

So let’s move on, says Mason – not through the forced destruction of market mechanisms by the working class, as anticipated by the left (‘over the past 25 years it is the left’s project that has collapsed’), but rather through the spontaneous, creative, self-organising activities of the ‘networked individuals’ that the working class has morphed into under neoliberal capitalism’s economic and technological pressures. In this, he believes, they will need to be supported by institutional innovation, imaginative state intervention, including the provision of a universal basic income, and internationally-agreed action to deal with the threats of climate change and the overhang of global debt.

Inevitably Mason’s ‘guide to the future’ is a lot sketchier and more speculative than his dense (sometimes over-dense, although vastly impressive in range and depth of reference) and rich analysis of the past – which has equally inevitably brought down a hail of criticism from sceptics. But two things are worth noting. First, in his closely documented The Rise of the Robots, Silicon Valley software entrepreneur Martin Ford strikingly arrives at the same end point from a completely departure: a basic guaranteed wage for all, he concludes, will be the only means of generating enough demand to keep the wheels of capitalism turning when work disappears into the machine. And second, the scoffers who ridicule Mason for being both apocalyptic and utopian are on no firmer ground themselves. It’s true that capitalism has shown a protean ability to adapt and transform up to now. But faith that it will do so again because it has done it before is just that – faith – and no more based on hard evidence than Mason’s scenario. Likewise the labour theory of value that underpins much of his economic thinking may be unfamiliar. But the assumptions that it rests on are actually no more outlandish than the perfect information, efficient markets and utility-maximisation of neoclassical theory, which has hardly proved itself a reliable guide to the real economy.

The truth is that Mason doesn’t know any more than anyone else what the world is going to throw at us in the next decades, and it is certain that at least half of what he proposes will be wrong – including, very possibly, the positive outcome (Evgeny Morozov, for example, takes a rather different view). Meanwhile, the range of possibilities that he presents is tantalising and invigorating, not least because he grants agency to individuals – if they (we) care, or have the willpower, to use it. You certainly won’t agree with everything here, and I lost count of the scribbled question and exclamation marks against the text. But Mason is asking the right questions, and in its heady interweaving of social, economic and political history, his book is a thriller. Even if you find yourself at an unfamiliar or even unwelcome destination, as is quite likely, you won’t have been bored on the ride.

Read my column in FT Business Education, ‘Why it’s time we rethink the nature of work’, here

My article in Professional Manager, 20 August 2015, ‘Will the digital revolution make us all poorer?’, is available here

The French newspaper Libération devoted four pages of a recent issue to an investigation of the idea of a minimum basic income for every adult. Among the surprises for an Anglo-Saxon reader, for whose politicians such a notion is so far out of mind as to be science fiction, is the discovery that in other countries it is very much on the map. The left-wing Spanish party Poderemos has it in its manifesto. The Finns are considering it, and the Swiss will vote on it in a referendum later this year. The practical Dutch are testing it in a pilot in Utrecht.

If Martin Ford is right, such initiatives, far from being a rush of Corbynitis to the head, are necessary and urgent. Rather than signalling the demise of capitalism, he suggests in his impressively researched and soberly argued book, The Rise of the Robots, a basic minimum wage may turn out to be the only way to save it.

Ford is a successful Silicon Valley software entrepreneur, which at first glance makes him an unlikely technological doomsayer. But unlike gung-ho contemporaries such as Ray Kurzweil who enthuse about the coming ‘singularity’, the moment when machine intelligence surpasses the human kind, and the achievement of immortality (sic), Ford concentrates on what is happening in the here and now to make a powerful case that this time it really is different; meaning that the threat in his subtitle – ‘technology and the threat of a jobless future’ – is very likely to come true.

The techno-optimists’ cheerful view of the economic consequences of technological advance is based on faith and precedent. They point out that since the Industrial Revolution first disrupted craft-working in the 18th century, each succeeding wave of progress has produced new markets and industries that generate more and better-paid jobs than they destroy. This is Schumpeter’s ‘creative destruction’ in action, a virtuous circle in which technological innovation drives higher wages and increasing demand that floats all boats. Better education will keep human capability ahead of the machines; economic growth will provide a steady flow of new jobs in partly-mechanised sectors and others that spring up to serve them.

But although many economists (and almost all politicians) are still parroting the old mantras about a return to growth, Ford notes that the positive relationships had started breaking down long before the financial crash of 2008, and substantially pre-dating the current round of techno-acceleration. (Ford writes mainly about the US, but in economics as in other spheres the UK can be relied on to imitate the transatlantic experience with a short time lag, if in slightly less extreme form.)

Thus Ford lists ‘seven deadly trends’ that have been ticking away behind the economists’ comfortable assumptions like a colony of deathwatch beetle: average wages stagnant since the 1980s; a shrinking labour share of GDP; declining labour-force participation (particularly among less qualified men); jobless recoveries (it took until mid-2014 for US employment to regain pre-crash totals, by which time the working population had increased by 15 million – in fact the US economy has put on no net new jobs this century); soaring inequality; a diminishing premium and growing underemployment for graduates; and the polarisation of the jobs market between well-paid full-time employment for the very few and part-time and freelance for the many – ‘uberisation’, let’s say.

Now take this malign dynamic and supercharge it with the most powerful general purpose technology (ie whose effects will leave no industry untouched) ever devised – one, moreover, whose advance is accelerating with the undiminished momentum of Moore’s Law. Machine intelligence is improving by leaps and bounds. IBM’s Watson supercomputer defeats human champions not only at chess, a bounded problem, but also at the game show Jeopardy!, a cryptic, unbounded one. Watson is now being deployed commercially. While Artificial Intelligence (AI) is still the ‘narrow’ variety, the ‘strong’ version, or Artificial General Intelligence (AGI), is now being vigorously pursued not just in research labs, as in the past, but competitively by ambitious, well-resourced giants such as Google, Facebook and Apple, pushed by powerful commercial incentives to make it work. Another AI, Artificial Intuition, is in the works.

A recent Forbes contribution (entitled none too subtly ‘Deep Learning And Machine Intelligence Will Eat The World’) could not have put it more clearly: ‘The effects of this technology will change the economics of virtually every industry. And although the market value of machine learning and data science talent is climbing rapidly, the value of most human labor will precipitously fall.’ Publications (reputedly including Forbes) already employ software to write news stories and reports; in a few years time 90 per cent will be machine generated, in one estimate. In many other industries, automation, says Ford, is simply ‘the logical next step’. Beware, he warns: if you’re working with computer software, you’re probably training it to replace you.

In a much quoted 2013 report, the Oxford Martin School suggested that 47 per cent of US jobs would be susceptible to computerisation in the next two decades. Later estimates raise that to 80 per cent. Given that the essence of computerisation is enabling more to be done with less, is it conceivable that new industries based on it will be labour intensive? Looking at early evidence from those avatars of the new economy YouTube (which had a value of $1.65bn when it was acquired with its workforce of 65) Instagram ($1bn and 13) and WhatsApp (a staggering $19bn and 55), the answer seems pretty clear. Uber and Airbnb just underline the point: for the first time new technology is not only creating fewer jobs than it consumes: it is also creating worse ones. The circumstantial evidence keeps flooding in. Thirty per cent of US science and technology graduates are currently labouring in jobs that don’t need degrees; most UK graduates are in non-graduate jobs. ‘The assumption that we will transition to a more productive … economy just by increasing the conveyor belt of graduates [the method used in the past] is proven to be flawed,’ says the Chartered Institute of Personnel and Development (CIPD).

So far, so not very good at all. But if the scenario follows Ford’s trajectory, the effects will be massively self-reinforcing through the resulting demand deflation. Ford quotes the following exchange between Henry Ford II and union boss Walter Reuther on a factory visit: ‘Walter, how are you going to get those robots to pay your union dues?’ ‘Henry, how are you going to get them to buy your cars?’

At the extreme, radical inequality is unsustainable in the most basic sense: by hollowing out the middle classes, the winner-takes-all economy becomes a contradiction in terms. There won’t be anything to take, because the very rich simply don’t consume enough to keep the wheels turning. Being partly the result of what happens when average consumers do keep consuming, but using debt rather than cash to do so, the 2008 crash should be a warning here. As Ford relates, companies are already abandoning the middle market to chase the 1 per cent of super-spenders; despite increasingly frantic advertising, the average US car is now 12 years old, a forerunner of the tangible consequences of rising inequality. In this context, the gathering cloud of graduate debt overhanging the US and UK looks increasingly ominous not only for individuals with increasingly uncertain earning prospects but also for the economy as a whole, while the policies that created it are revealed as the monstrous false economy they seemed to many at the time.

This is the background against which Ford sets out his proposal for a universal minimum basic income. Ironically, the original progenitor was the English radical Tom Paine, who advocated it as a blow for social justice against the cruelty of emerging capitalism. By contrast, for Ford it is primarily a prop to keep that system going. As he concedes, the idea is controversial, with weighty considerations on both sides. Yet the need for solutions may be even more pressing than he thinks it is. Ford didn’t foresee another emerging result of global inequalities, the rising tide of immigration. And like almost every other commentator, whether optimist or pessimist, Ford has internalised, and thus leaves out of his reckoning, the most deadly trend of all: the pernicious incentives at the heart of today’s shareholder capitalism.

This is the first great wave of technological evolution whose justification is not that it benefits humanity but that it benefits shareholders. The lesson of the last 30 years is that Investments driven by self-interest and shareholder value – where the benefits are supposed to accrue to one group in society – do not produce a generalised increase in wellbeing, because they are designed not to. As Jeff Pfeffer succinctly puts it, ‘Economic performance and costs trump employee [and societal] wellbeing’. Under today’s incentives, investments in accelerating technology will just destroy more of it. To be clear about this, consider a quote from the founder of a start-up planning to automate the production of customised gourmet hamburgers: ‘Our device isn’t meant to make employees more efficient. It’s meant to completely obviate them.’ Or this from another start-up entrepreneur warning that executive jobs too are in the firing line: ‘It will not be possible to hide in the C-Suite for much longer. The same cost/benefit analyses performed by shareholders against line workers and office managers will soon be applied to executives and their generous salaries’. Oh yes, the name of this start-up: iCEO.

At this point, as Ford notes logically if bleakly, at the end of his book, falling demand may run out even for further automation. It might. But it would surely be unwise to wait to get that far to find out.

****I’m away at the moment: next piece beginning of September****

And another thing…

The pother over productivity has reached such a pitch that I make no apology for returning to it.

In one of the latest contributions, the newly-Japanese-owned FT recently carried an article suggesting that the solution for the UK’s low-productivity problem was to cancel August – in other words, everyone should take less holiday. I think it was meant seriously.

It brought irresistibly to mind W. E. Deming’s acerbic, ‘Having lost sight of our objective, we redoubled our efforts’, instantly putting a finger on what’s missing from the productivity debate: it’s all very well working harder, but to what end?

‘Productivity’ is about ‘efficiency’, and there is no argument that UK productivity/efficiency is historically poor, lagging that of many of our national competitors. But efficiency is a crude proxy for a more important and profounder measure, which is effectiveness. Like GDP, productivity is a measure of activity in general, irrespective of whether it is useful and beneficial for society or not. It measures outputs against inputs and is about means – doing things right. Effectiveness measures outputs against objectives and is about ends, or doing the right thing. Unless it’s related to the right thing, productivity is of secondary importance. As Peter Drucker decisively summed it up, ‘There is nothing so useless as doing efficiently that which should not be done at all.’

The economic story told by effectiveness differs in important ways from the one derived from the productivity figures. For example, the official narrative puts much emphasis on improving productivity through indirect supply-side measures such as investment in infrastructure and education, and hiving off as much activity as possible to the supposedly more efficient private sector.

Of course, functioning infrastructure, including education, is essential. But it has nothing to do with organisational effectiveness, which is not a private-vs-public-sector issue: it is one of system design. All over the economy people are being beaten up to do more efficiently stuff which shouldn’t be done at all – either because they are attempting to do the wrong thing righter, which, as systems guru Russell Ackoff points out, just makes them wronger, or because they are redoing something that wasn’t done or wasn’t done properly the first time round: ‘failure demand’, in John Seddon’s term.

The dirty secret is that the NHS and many public services are stuffed full of failure demand – in some cases 60 to 80 per cent of contacts are repeat calls from folk who haven’t had their problem solved the first time. But so too are the customer-service departments of banks, phone companies and other utilities which measure their activities in terms of efficiency (x number of calls per hour, x number of rings to answer the phone) rather than effectiveness (the overall time it takes to fix a customer’s problem). In other words, measured against their purpose, whatever the productivity figures say, they are hopelessly ineffective.

At a stroke, this nullifies a second part of the official narrative: in shorthand, the death spiral of public services. The conventional story is that services are unsustainable, groaning under ever-increasing demand and expectation, necessitating constant cost-cutting to make them more efficient. But this is a travesty of the truth. What they are groaning under is the weight of citizens failing to get routine problems fixed by a system that is designed against cost, not to meet predictable demand.

Nor is it true that there is an insurmountable resource crisis; or rather, there is, but it is caused by a work design in which jobs are so tenuously connected with needs that they mostly make things worse rather than better. That includes anyone working to internal service agreements or standards, having to meet numerical or time targets or quotas, managing demand (rationing) rather than meeting it, working in back offices, shared-service and most contact centres; central specifiers, commissioners, inspectors and regulators; and anyone managing the same, basically policing other people and administering the rules. That’s a lot of people.

The bad news is that all these are what anthropologist David Graeber has termed ‘bullshit jobs’, make-work employment offering no meaning or pride (no wonder levels of engagement are so dismal). The good news is that any improvement (doing the right thing, however imperfectly at first) brings a double benefit, reducing wasted work, and therefore cost, on one side, while freeing up capacity to do more on the other. Climbing morale as people are reconnected with a meaningful purpose adds a third whammy. So yes, there is a resource crisis – but it’s a crisis of management effectiveness, not individual productivity.

I recently came across an intriguing concept called ‘Eroom’s Law’ – Moore’s Law backwards, if you haven’t got there, and it applies to processes that unlike computer computer processors get slower and more difficult over time. It was first applied to the seemingly inexorable slow-down in new drug discovery, but it also usefully illustrates what’s happening to management as it proliferates and slows under its own friction (I wrote a bit about how it happens previously here). Unless we can prise management out of the grip of Eroom’s Law, any increase in overall productivity will will be more than eaten up by the rising tide of bullshit and a corresponding decrease in effectiveness.

To be blunt: how can you be customer-led if customers don’t know or can’t tell you what they want themselves? From the company angle the Holy Grail is a market game-changer: the automobile to replace the horse, say, or an iPhone, or an iTunes or Spotify. But if as a customer you’ve been thinking in terms of more rapid equine transport, a telephone that’s better at talking on, or a larger collection of CDs, what do you make of a machine with four wheels and a clattering engine, a supercomputer with a touch-screen, or a virtual jukebox hosted in something called ‘the cloud’? Round the other way, how come despite decades of experience, pollsters still find surprises and contradictions in people’s actual behaviour, even when polled the day before an election about how they are going to vote?

The answer to the questions that came back from an intriguing Foundation Forum on 10th June was that human reactions remain more or less as infuriatingly difficult to call as ever. We have a lot more data about why we react as we do. But while that helps us understand the degree of our unpredictability, knowing more accurately the scale of the problem doesn’t make it easier to get business decisions right. To paraphrase Lord Leverhulme, we know that half our knowledge about the way people will jump in any situation is wrong: we just don’t know which half.

‘If you’d told me in 1987 that I’d employ 100 filmmakers in 2015, I’d have wondered why I’d ever do that, or that I’d employ 20 economists’

The challenge for the research industry therefore remains as great as ever. As Ben Page, chief executive of pollster Ipsos MORI, noted, while some of the techniques are remarkably unchanged – companies still knock on doors, do telephone interviews, and send out postal surveys – they have deepened. ‘If you’d told me in 1987 that I’d employ 100 filmmakers in 2015, I’d have wondered why I’d ever do that – or that I’d employ 20 economists’. That reflects more sophisticated attempts to tease meaning out of the information gathered, with a discernible shift of emphasis from asking questions to watching and observing what people do. Today’s key trends, says Page, are speed (clients want reports in hours, not weeks), mobile (location recording, instant selfies at the breakfast table) – and the dawning realisation that since subjects aren’t completely rational, it’s not enough just to record what people think they do.

That means that no one research tool is adequate on its own. ‘Instead what we’re seeing is layering of these different techniques, so clients will be looking at a whole range of different data sources. And as we better understand these things using all the techniques at our disposal we’re getting a much better and richer understanding of human behaviour’, he said. So it’s not a question of either intuition or research, but both/and, and a lot of other things besides. Indeed, intuition remains a powerful force – ‘an Ipsos MORI chief executive wouldn’t be advising anyone to scrap research and just listen to instinct, but it’s amazing how many clients pay millions of pounds to evaluate an ad campaign and then cheerfully ignore the data and go with their gut feel.’

To understand how people work ‘you need research, you need data and you need psychology’

Marc Michaels, the second panellist, agreed that to understand how people work ‘you need research, you need data and you need psychology’. And as someone initially recruited to set up a government direct-marketing unit and then more generally to work on changing behaviour – persuading people to eat more sensibly, give blood, join the army: a tough brief – he is clear that in making research and data actionable, the new findings of psychology and behavioural economics are critical. It’s not that we lie, he says; but as Daniel Kahneman demonstrated in Thinking, Fast and Slow we each have two brains, a System 1, ‘Homer Simpson’ organ for instinctive, holistic, and instant decision-making, and a System 2, ‘Spock’ brain for more analytical, deliberative, demanding thought. There are cognitive biases that undermine strict ‘rationality’ but which also give opportunities to ‘nudge’ people towards certain choices or behaviours by fitting the way they are engaged to the innate predisposition to respond in one direction or another.

Thus there’s a general human tendency to fear losses more than to value gains (‘a bird in the hand is actually worth two point five, sometimes even three, in the bush’). The slacker, Homer Simpson brain will try to get away with answering an easier question than the one asked. Big, complicated issues are often avoided, so ‘chunking’ them down is likely to win a more positive response. As Stanley Milgram’s famous 1960s experiments showed, people respect and obey authority, sometimes to a frightening degree. So, in one celebrated example, the Department of Health and COI recruited Anne Diamond, a respected and well-loved newsreader who had suffered a cot death, to counter the grandmother-sanctioned traditional wisdom of putting infants to sleep on their fronts and persuade young mothers to sleep them instead on their back or side. The result: a reduction in cot deaths of 70 per cent. But doing it needed authority to fight authority.

In a sense, research has gone full circle, observed Clive Humby, co-founder of data-led research business dunnhumby and now Starcount, which uses social media and ‘fan science’ to craft brand influence strategies. ‘Really, what we’re really talking about with data is understanding customers through the things that we observe. We’ve heard about watching them through videos, asking them questions, and obviously looking at what people physically do close-up and the transactions they’ve made using information. It’s gone through a complete revolution’.

Humby is credited with the line that data is the new oil, and he drew two important comparisons. First, the gusher in its raw state has little value, only becoming usable when it is processed into something else. Data is the same: ‘Data is everywhere. I’ve got 147 devices in my house that have their own IP address – smart TVs, a lighting system, computers, phones, all those items are generating data about you all the time. The data on ourselves generated in the last day, exceeds all the data generated in a year 12 months ago. So the real challenge isn’t in collecting data any more, there’s far too much of it – it’s making it useful’. So it’s the algorithm guys, the pattern-identifiers that are the stars, the car designers building on the potential of oil.

‘The real challenge isn’t collecting data any more, there’s far too much of it – it’s making it useful’

Yet solving one problem often just reveals another. Made usable, into petrol for instance, oil becomes volatile. So too with data, which turns not just volatile but nuclear when it collides with privacy. Benign nudging and behaviour-recording with consent are one thing; but what about using your shopping list as a basis for insurance premiums? Or a real example from Tesco: ‘One of the most important correlations we found in terms of data we could have commercially exploited, was the one between £4.99 Chardonnay and condoms. But we never acted on it. The reality is that just because you know, doesn’t mean you should. And that is the dilemma we’re all facing’: Is it cool or is it creepy?

The dilemma can only intensify with the rapidly emerging internet of things, in ways that have barely yet been registered. Suddenly the issue is no longer the quantity of personal information being given away to a faceless corporate. ‘We think about privacy in terms of our big corporate systems and frontline operators who talk to customers’, Humby pointed out. But actually the data is available to developers, app people and potentially everyone in the organisations that has access to it. The people who repair your car know everything about where it has been, how fast it was driven, how long it was parked. How easy would it be for someone to get this and use it for something unforeseen and with bad intent? ‘Once that happens, everyone becomes a possible liability. And we have to really worry about that as leaders in organisations’.

The paradox of research, as with most things human, is that the more we know, the more complicated it gets

The paradox of research, as with most things human, is that the more we know, the more complicated it gets – and a simple scientific synthesis seems as far off as ever. At least for the time being, it’s humans that rule, not algorithms. Noted Page: ‘It’s taken some time, but I think the industry is getting there. One of the things that’s holding us back is that sometimes we’re conservative, and our clients are just as conservative because they’ve been tracking data the same way for 30 years, and it’s consistent and tells them things’. His advice is: relax a bit, be creative, and remember the lesson of Alex Salmond – who on the basis of extremely expensive US analysis through social media knew for certain that the Scots would vote for independence, never mind the opinion polls.

‘People aren’t rational,’ summed up Michaels. ‘When they tell you they want to do something, you may see from the data side that they’re doing something, but you’ve got to think about what is going on there. You can think data, but you need to talk human’.

The Foundation’s thoughts

Four of the most significant points which emerged for us were as follows:

• Collaborative businesses are succeeding because they bring at least three useful characteristics together in a way that reinforce each other. Any new and better business model tends to do this, creating a virtuous circle that is different enough to the incumbents’ for it to be impossible to copy with a simple adjustment:

• To get to a good understanding of what people think, feel, and crucially, do, can take the application of all of the approaches described above. On the evening we talked about triangulation, using market research to understand the landscape, then in the areas of interest conducting deeper exploration. This might use more extensive real-world data, and the vastness is made useful by developing and testing hypotheses based on human understanding that respects the instinctive ways we often act. Another way to describe the process is one of detective work – an overall conundrum to be solved, and lines of enquiry established around the possibilities. Each then explored creatively (what could be going on here?) then challenged to eliminate as much as possible from enquiries using data, further specific research or conducting experiments.

• As Clive reminded us, there is a rear view mirror issue with data. It can tell us what’s happened, and used well it can give us insight into why. But it can’t predict the future. Which might make some of the big data investment going on right now look a bit optimistic

• The vastness of the data we each generate creates real ethical issues that aren’t currently being addressed. It is much easier than we realise to share information on everywhere we’ve been, everyone we’ve spoken to and a fair bit of what’s been exchanged with all sorts of organisations and individuals that we might be wary of if we sat down and thought about it. As we heard, modern cars contain information on where they have been and how they were driven, all easily accessed by your local car dealer, the police or your insurance company… or anyone who knew how to hack and steal it. We often allow apps to get this kind of information from our mobile phones, because we click ‘allow’ and because the Apple Ts&Cs are, in Clive’s words, longer than Shakespeare’s The Tempest.

• Our human intuition isn’t just at the end of the telescope trained on customers. The users of insight have just the same biases, from the more entertaining ‘I don’t care what the research says, we’re running the ad’, to the more important problems we find with inconvenient market research findings getting short shrift from a leadership team trapped in a world they see from the inside of their business looking out. It can be useful to see the conclusions from insight work as the start of another challenge, giving it the impact it needs to get the organisational response it requires. For example, getting leaders speaking to customers themselves so they create their own stories and beliefs in line with the bigger picture.