“Absence of evidence is not necessarily evidence of absence”, or so we
have been told. Just because the water seems fresh and clear doesn’t mean the liver flukes aren’t there. We shouldn’t get paranoid over that, but we ought to still boil the water before drinking or make sure the water gets filtered or otherwise rendered as safe as we can get it to be. That doesn’t mean we should never scoop the water up and drink it if we happen to be in dire straits and have to move somewhere else fast.
People, and that means organizations which have people working in them, have a tendency to ignore operational and living protocols which call for self and mutual checks. “If it ain’t broke, don’t fix it” or so I have been told many times over the years by my seniors and betters. People who have diabetes, who need to monitor stuff in their blood and have their insulin injections at hand, appreciate the need for “own self check own self” better than otherwise healthy others. Well, maybe those with leprosy understand very well the need for VSEs (Visual Surveillance Exercises), making sure that no part of their bodies has been damaged and might move on to being gangrenous. It is always wonderful to have the support of those in the medical profession who help stop us from dying, at least temporarily, and who render advice on how to stay healthy and happy based on all the information, data and experience garnered over many years. That does not mean we can stop being our own best doctors.
I happened to come across a very interesting video “From the Board Room to the SIT Room” some time ago. It was moderated by the Hoover Institution’s Dr Amy Zegart and featured a panel of “two Johns and a David”. What I found so fascinating about it was the portrayal of our exponentially increasing requirement for having more and more layers of instrumentation to help us make better decisions and even to become our default reflex actions. What happens when the dashboard we face is chock full of information and we don’t know whether we can actually trust it? What happens when our systems initiate a “fire back” response and we realize too late that the current situation has all the red lights that the AI (Artificial Intelligence) recognizes and has been programmed to react to, but the whole context is different, and we should NOT have “fired back”? No, this is not science fiction, it’s happening now.
In the excellent old film “Signposts Aloft“, the point that pilots need to trust their flight instruments and not the seat of their pants is hammered home. Where we have strayed, methinks, is when we stop relying on instruments to be instruments and start demanding that they become “Sages on Stages” and guide us in what do we do next.
I’m not too concerned about Absence of Malice. I’m terribly concerned when we surrender our humanity to the work of our own hands. Just because our systems have not detected any penetration that has our very expensive infrastructure compromised does not mean we haven’t BEEN penetrated. I’m even more concerned that what seems to have been a penetration was actually a pernicious system design flaw completely our own and we are in this precarious state through the fault of no one else.