Unpredictable Humans

In the kind of computer games where you, the human player, compete against the computer's artificial intelligence (AI), you have a huge advantage: you're human. Humans are unpredictable, unlike the predictable AI. In Civilization VI, on the highest difficulty level - Deity - playing single player against computer (AI) opponents, you can still beat the AI even with the huge advantages they are given by the game. Defending your city against massed ranks of AI units, you can predict what they will do but you - the unpredictable human - can outwit them by doing something crazy. City nearly dead to an AI unit? Send out a builder to distract the unit; they will take the builder, I will then attack and they have gained a builder but lost the war. The AI doesn't think strategically.

Which is my problem with driverless cars. I was driving out of Tesco this morning, about to turn right. Being careful. Car approaching from the right. Signalling left, to go into Tesco. That's OK, I can go .... whoa! The other car is going straight on, not into Tesco! Still signalling. Screeching of brakes, near collision. OK, exaggerating a little. No actual collision although a few words exchanged. I assume that the driver wasn't aware that their indicator was still on but maybe it was just a last minute change of plan.

How would my (mythical) driverless Tesla deal with this unpredictable behaviour? I imagine the Tesla AI can see the flashing indicator and assess the probability of the car behaving as expected, i.e. turning left. Of course, my Tesla doesn't know whether the other car is also driverless - therefore predictable - or human. Therefore there is a knowledge gap, which is a problem for an AI. My guess is the Tesla would sit there waiting for a gap in the traffic. This is the ultimate safety play but this is Tesco, so it's possible that there's a long stream of traffic, all turning left and with their indicators on. Because they are human, some might not be indicating but still intend to turn left. We'll be there for ages.

Then there's the potential moral dilemma. I read a really interesting article on this on the BBC's Science Focus website, positing for instance the decision making a driver might have to go through in a situation where every possible decision will result in potential death but the lethalities are unequal - adults vs children, ten passengers vs one, others vs me. For humans the speed of the situation is too much for us and means we react purely on instinct, with decisions which will not be the same for all of us; we are unpredictable. But the driverless car AI can process all possible outcomes virtually instantaneously and make a 'moral' decision to pursue one (probable) outcome.

When driverless cars were first mooted, my reaction was "terrific, I can read a book while my car takes me safely and speedily to my destination". I still think that, but with one proviso - that all the other vehicles on that particular stretch of road - perhaps a specially built highway - are also driverless and (preferably) powered by the same software (or industry standard algorithms). While we are (for my lifetime and for decades after, I suspect) in a hybrid situation where some cars could be driven by predictable AI and others by unpredictable humans, I'm against the idea.

What do you think?