In his blog, Luca Chittaro explains the paradox of automation in critical systems by elaborating the Lisanne Banbridge's paper on "Ironies of Automation". Basically, nowadays computers take care of automation of complex control tasks (e.g. in aircrafts), while human are requested to monitor the correct behavior of the system, and switch (when possible) to manual control in case of detected failures. Monitoring is even a more boring task than controlling instrumentation. Moreover, intervention in case of failure is more difficult and critical than controlling the system in normal conditions.
I believe that the role of computer is exactly the opposite than the current practices. Computers should "silently" monitor the interaction between users and the physical devices and take care that everything is done safely and correctly. When something wrong is detected, they should first warn the user and ultimately take control of the devices in order to re-establish a safe state.
I advocated this perspective in human-computer interfaces in my recent work on Kinetic User Interfaces where I propose an interaction pattern called "continuous tracking". In this pattern, the user performs some motion activity within a given context (e.g. in a place). The system's interface continuously monitors the activity and pops up providing relevant feedback (e.g. warnings, alerts, information) when something unusual happens (e.g. anomalous, danger, new). In the normal case, the interface stays in the background providing only minimal feedback in order to tell the user that the monitoring is done correctly.
This pattern has been implemented by Pascal Bruegger in a test scenario for paragliding assistance: the UbiGlide project and showcased at UbiComp 2007.