This is circular reasoning, since we never established that they had goals or agency in the first place. It’s not as if they did, then I got burned and took it back! Software is a tool, so it functions as an extension of the mind of the programmer.
My experience is that people who assume that computers have their own motivations are those who don’t know how computers actually work. Programmers (I am a novice) tend to be painfully aware of how much work is involved in getting a computer to perform some apparently “basic” task, and know that it is far from being automatic - as might appear to someone who runs readymade code with no consideration of the time and effort required to implement it.
That a system’s stated goal appears orthogonal to your interests is not a good reason to think it is safe. It simply means that you - your existence/life/well being have absolutely no effect on what actions the system takes. Paperclip maximizers are a whole lot scarier and more plausible than actually malevolent AI (unless a military organization is the first to build general AI and intentionally makes it malevolent).
A human’s definition of safety is also relative to their own values and goals. Is an unthinking desire to perpetuate my existence for arbitrary reasons really a sound goal, compared to a reasoned one? If I will assuredly die, is my death truly safe/unsafe in any objective sense? This is one of the biggest mistakes humans make - assuming that their values and goals are (or should be) somehow universal. If people are basically programmed to do this, how are they not merely nanotech automata themselves?
This is what I think the revolutionary power of computers really is: to allow people to analyze the patterns of automatic human behaviors, including and especially those which would be too innate for them to otherwise be aware of. It’s really the only development which has had the power to help humanity to definitively see beyond its own cognitive limitations.
This topic was automatically closed after 5 days. New replies are no longer allowed.
