The Vigilance Problem

The moment I heard about self-driving cars I thought of an issue, but I haven't heard anyone talk about it until recently.

The issue is that a car self-drives in good conditions and then hands over control when there is a problem or a situation that the car can't handle.  The handover is problematic.  First, the driver's attention is questionable at that point.  Second, how much time does the driver have to go from monitoring mode to crisis control mode?  Even if the driver is vigilant, can the driver make an evasive move in two seconds?  Will the driver know how much room there is on either side or behind the car?  Third, how good is the self-driving AI at knowing when it's working or not?  How much warning will a driver have to take over?  If you get a warning beep 2 seconds before an accident would happen, but it would take you 3 seconds to take control and avoid the accident, that's a fail.  Moreover, if the self-drive thinks it's right but the human driver is overriding, it's essentially fighting the self-drive.

I've just learned that the first two issues are called "The Vigilance Problem."

the problem is that people are just not great at paying close attention to monitoring tasks, and if a semi-automated driving system is doing most of the steering, speed control, and other aspects of the driving task, the human in the driver’s seat’s job changes from one of active control to one of monitoring for when the system may make an error.

Handing over control is complicated.  For example, the case of doctors handing over care at the end of a shift.  Complications happen more in this situation than in any other in the hospital.  Expecting a driver to take over control only in abnormal situations and under a very stressful time crunch is a recipe for disaster.

The three problems are: (1) Can the driver maintain good attention when monitoring?  (2) Can the handover in control be seamless when it will only be done in adverse conditions? (3) Will the self-driving system know when to disengage, notify in a timely manner, and not interfere with the transfer process?

These problems can be applied to any area with a Principal-Agent Problem.