The Reason for the Human

There used to be a man who walked the length of a stopped train with a hammer.

He struck each wheel and listened.

A sound wheel rings differently from a damaged one. To the trained ear, the change could reveal a defect serious enough for the vehicle to be taken out of service. The wheeltapper also checked axle boxes for heat as he moved along the train.

It was simple work only in appearance.

His judgement mattered because he could perceive something consequential, at the place and moment it needed to be perceived. The information could not be sent somewhere else first. The person and the problem had to meet.

Much of that work eventually disappeared. Inspection improved. Ultrasonic techniques could detect defects that a hammer could not. Trackside monitoring could identify problems as trains passed.

The information no longer required the same person to be standing beside the wheel.

There was no argument about it.

I wonder whether artificial intelligence will eventually force organisations to confront the same question.

For a long time, there have been practical reasons for placing humans close to consequential decisions.

They possessed information others did not. They understood local conditions. They recognised patterns built through experience. They could interpret circumstances that travelled badly through organisational systems.

Proximity gave them an informational advantage.

Increasingly capable systems may begin to erode it.

A machine can already bring together more information than an individual could reasonably examine. It can detect patterns across thousands or millions of previous cases. It can compare possibilities and produce predictions at the point where something is happening.

On the situations it was built from, its recommendation may simply be better.

That creates an uncomfortable question.

Why was the human there in the first place?

If the answer was simply that they knew more, that answer may not survive.

But perhaps knowing more was never the whole reason.

A prediction can inform what is likely to happen without determining whether that outcome is acceptable.

An optimisation can identify an effective route towards an objective without determining whether the objective remains the right one.

A recommendation can become increasingly reliable without acquiring, merely through its accuracy, the legitimate authority to determine what ought to happen.

That distinction is relatively easy to maintain while the human is also the most knowledgeable participant in the system.

It becomes much harder when they are not.

And perhaps that is where the argument becomes more uncomfortable still.

If the human contribution to a consequential decision is not primarily superior information, it does not follow that a human must remain present at every instance of that decision.

The judgement may belong somewhere else: in establishing what the system is permitted to do, determining which consequences are unacceptable, and retaining the authority to change those conditions when they no longer hold.

If that is true, making the case for human judgement properly may not result in more humans in loops.

It may result in fewer.

But the humans who remain may carry considerably more consequential authority.

And answer for outcomes they will never individually see.

That returns us to an earlier problem.

A human can remain visibly present long after the reason for their presence has disappeared.

We have seen that horse before.

Perhaps increasingly capable intelligence will therefore force a distinction organisations have so far been able to avoid.

Not between decisions made by humans and decisions made by machines.

Between where intelligence is useful and where human judgement is necessary.

Those boundaries may not be in the same place.

And keeping a human where they are no longer needed may be as serious an error as removing one from somewhere they are.

Question for the Field

Find one consequential decision in your organisation that you believe should remain with a human.

Now remove the easiest reason for keeping them there.

Then ask:

What is the human there for?

And if you can answer that, ask one more:

Do they need to be there for every decision?