Not to mention God, with the whole “you have a choice but if you don’t choose to love me I’ll torture you forever.” What a dick.
Yup, and I don’t think that problem is going to go away. A lot of stuff that we might consider hallmarks of intelligence, such as achieving your goals without murdering anyone, are actually carefully evolved and utterly arbitrary social constructs. As far as I can see there’s two ways around the problem:
- A human being must check every solution that an AI algorithm generates, to ensure that it is a good solution in the context of human society, in which case what’s the point, or:
- The AI must be thrown fully into the human context, with a corporeal form and access to the normal human system of development through rewards and punishment within human society, in which case what’s the point?
I suspect that AI won’t get “better” than humans because there is no “better,” it can either make decisions that make sense to us or it’s doing it wrong. And trying to emulate human thought with binary computation is just an incredibly inefficient use of electrical signal. The world’s fastest computer takes up 720 square metres, uses 24 megawatts (1.2 million times as much as a brain), takes something like 40 minutes to run a low-fi simulation of 1 second of brain function, and Moore’s law ain’t what it used to be.
My prediction is that by the end of the century, our ability to emulate the brain on digital hardware will be surpassed by our ability to grow organic brains to spec.
