Fair point, though in this particular scenario the AI would be more likely to be incentivized to lie to convince the operator to issue the “don’t kill” order, since not killing is simpler than killing (since it’s the default result of inaction).
Wholeheartedly agree. We should never design AIs that can actually kill people, because we will screw it up.
Still, AI isn’t going away, so we will need to learn how to use it responsibly. You don’t have to build a murder machine to do real damage with a poorly designed AI model. While I suspect this particular experiment was intentionally designed to highlight some of these problems, if it was a serious attempt to model an AI killer, their approach (and especially their follow up patch) was incompetently flawed.
Exactly right. And more than negative points, attacking your own side should completely void the game. The problem is that for ages now “friendly fire” and “colateral damage” have been an integral part of the equation in war simulations.