I love me some SMBC, but the immediate danger from machine superintelligence isn’t it becoming self-aware. There’s a good chance we’ll eventually give rise to self-aware artifacts decades hence. But right now we’re weaving algorithmic webs of non-self-aware machine intelligences we don’t really understand and over which may not be able to effectively maintain control. I myself work in one small corner of that vast interdisciplinary enterprise.
Our goal as a society and within our technology industries should be to research ways to build and evolve machine intelligences that aren’t hostile or indifferent to our own goals, and which answer neither to only a small fraction of humans or to none at all. This isn’t some distant esoteric concern that can be dismissed as receding with the horizon. We’re catching up daily.
