Marvin the Paranoid Android

As regular readers will know, I spend quite a lot of my time in that uncomfortable zone between contemplating the giddying possibilities that emerging tech seems to offer us all and the horrors of a truly dystopian future ruled by robots and other tech that no one (including me) understands.

Sometimes I attempt a grown-up discussion about what the implications (social, environmental, professional, political) of rapid, widespread, or uncritical adoption of a new type of tech might be.

Because it strikes me that the tech sector has plenty of people already making the case for whatever technology they want to promote (sell), it often falls to me to suggest that we might also want to consider the downside. It’s not a popular move. One colleague even accused me of being a Luddite, simply because I had the temerity to say that I wouldn’t want to have to work with a particular technology in my workplace! – my specific objection was that it (I won’t name the tech concerned) would involve excessive monitoring and a loss of autonomy and perhaps result in my professional judgment being replaced by a one size fits all “protocol”.

I won’t be deterred though. Tech has to earn its place, just like anything else. Not all technologies out there (or ones yet to be invented) are beneficial. Not all are worth the money we are expected to pay for them. And not all will contribute to a better workplace, much less a better society. AI is of course the biggest threat. (Don’t take my word for it; the late Professor Stephen Hawking famously asserted that  “The development of full artificial intelligence could spell the end of the human race.”

 I’ve also read my Isaac Asimov, so don’t tell me the laws of robotics are foolproof!

A machine that learns autonomously poses an obvious threat. The degree of threat will vary according to context and of course, there is already lots of technology that might fall under the heading ‘AI’ already influencing our daily lives (for better and for worse). It’s a question of scale and purpose. What applications do we want AI for? What do we not want it for? And what happens when things go wrong?

Machines go wrong all the time. A machine that has “intelligence” can also, potentially, think itself into all sorts of problematic concepts and states of mind including perhaps dysfunctional states of mind. Marvin the Paranoid Android is a case in point. Blessed with a “brain” the size of a planet. Marvin can solve almost any problem any mere human might want to set him. But it’s all too tedious for him. Surely, he thinks, I’m worth more than this… depression and paranoia set in.

Happily, Marvin is not a power-seeking android. He quietly slips away, glumly, saying “Goodbye, Arthur”.

But what if Marvin had been more of a Terminator? It would have been less of the gloomy, but harmless, “Goodbye, Arthur” and more “You’re terminated ******!”.        

How do we prevent that from happening?

Well, I’m going to suggest that we all become slightly more demanding customers. Let’s all keep a cool head. What kind of workplace do we want? What do we not want? As we digitise more and more processes (if we must) then let’s make sure we know what improvements we actually want. There are lots of “solutions” out there looking for (often imaginary) problems to which they are the answer,  but what is it that we want to see improved in the workplace? What tasks would we readily leave to an algorithm? Which tasks are we determined to keep under our individual, professional, control?

So what I’d suggest is that we all spend a little more time thinking about what we want, or, in procurement-speak, what our requirements are. Only when we’ve got a handle on those will we be ready to engage with the market and then the market (Big and little tech) can shape its products and services accordingly.

Tim Heywood, solicitor, FRSA is a partner in gunnercooke llp specialising in tech; data, cyber, and procurement.

This article is not legal advice. If you would like advice on any of the issues raised please contact Tim Heywood at [email protected].

The views expressed here are personal and are not necessarily those of Gunnercooke llp.