I think our disagreements are narrow. I don’t think that the problems you describe are new. We are already faced with complex self-configuring systems (see: any operating system install), beyond the capacity of any human to comprehend. We deal with this complexity through careful and rigorous testing and bound setting. We also encounter systems where we must organize the work done by unconstrained intelligences (viz, other humans),and we have practices for dealing with those too, i.e. socialization, laws, mores, etc.
What is new is the vogue of believing you can set loose complex, fault-intolerant agents on your process and expect them to magically perform well. But this is also an ancient habit with a name: idiocy. It will soon give way to reality.
