But there is an important additional clarification needed. All AI (at this moment in time at least) is always understandable to a human. The very human (or humans) who created the AI (or can we just call it the algorithm).
The point of AI ‘legibility’ is to make[quote=“Javier_Agudelo, post:20, topic:100385”]
the why understandable to a human
[/quote]
not involved in the creation of the system.
AI ‘legibility’ is a concession by Software Engineers to the rest of humanity. Overcoming their reluctance to share their insight into the ‘why’ with mathematically challenged mortals . The other aspect is data transparency, but that is a can of worms.
Yes, the royal we of programmers might do, we humans mostly don’t.
To me as a person who studies language rather than numbers, your use of ‘we’ is a neat demonstration of the AI ‘legibility’ problem. Your use seems to shift from embracing all of humanity to referring to those mathematically literate. Not a critique, merely an observation. It’s about Engineers making the processes more transparent. A bit like the Centre Pompidou revealed the inner workings of architecture:
I agree ‘reasoning’ is the wrong term as is ‘legibility’. Mostly we are talking about transparency of process: Ensuring that people whose lives are impacted by the algorithm have the tools and access to the tools to understand its working (to a reasonable degree).
I think that what they’re more concerned about is machine learning.
So, to go back to the sarcasm story a few days ago. If you use machine learning and those data sets to train your algorithm to detect sarcasm, and then it flags the following comment as sarcastic:
I’m sure the Republicans have our best interest at heart. Really.
…Sure, the programmer could say “The algorithm classified this comment as sarcastic because it is more similar to the comments in the ‘sarcasm’ data set than the ones in the ‘not sarcasm’ data set.”
More useful would be “The algorithm classified this comment as sarcastic because the word ‘sure’ was italicized and the comment ended with a one-word sentence of ‘Really.’ These are more common among sarcastic comments than earnest comments.”
That’s the part where they’re having trouble with legibility: not the human-generated bit of “Okay, these are the things that we want you to learn to distinguish from each other,” but the machine-learned “Okay, this is what I’ve learned about how to distinguish these things from each other.”
Question: Isn’t the machine-learned bit determined but the human-generated bit?
It’s one of the key points Cathy O’Neil makes in Weapons of Math Destruction. The data we feed into the system determines the data we get.
In the final reckoning machine learning systems are closed systems. We keep hoping in spite current evidence human systems are open…
e.g. I am trilingual. As a consequence spell checkers don’t work for me. Spellchecker are trained to learn from one language. If as a human you routinely use more than one language the system is completely unable to differentiate and learns gobbled goo the data becomes too confusing. Particularly interesting is the inability to cope with verb conjugation. English you hardly conjugate, German, Hungarian conjugate verbs in present tense yet my phone keeps insisting that I use the wrong verb form because its anglocentric mind can’t imagine the manyfold variations of a single verb are all correct.
Using an Apple product in a language other than English is a great demonstration of the maxim that what you put in determines what you get out.
[quote=“nojaboja, post:23, topic:100385”]
Isn’t the machine-learned bit determined but the human-generated bit?
It’s one of the key points Cathy O’Neil makes in Weapons of Math Destruction. The data we feed into the system determines the data we get.
In the final reckoning machine learning systems are closed systems.[/quote]
That doesn’t mean that the machine-learned bit is necessarily human-intelligible.
For the simplest allegory: Given the public key and plaintext, the programmer of an encryption algorithm can easily show you how to turn that into ciphertext.
However, even given the public key, that same programmer will be helpless to turn a different ciphertext into plaintext.
The fact that it was produced by a machine through code written by a human doesn’t mean that a human will be able to read the output, even given most of the inputs.
Once we start get into machine-learning-created machine learning code, it’s going to get even more difficult.
You are actually touching on another much more important problem, transparency. Legibility is a pure technical problem, but transparency (I like the way “make the processes more transparent” sounds) is mostly about regulation and laws. The two are related but are not the same.
The biggest problem is that people don’t get any access to the algorithms because they are proprietary, but this has to do with the craziness and idiocy of modern IP laws. AI makes the problem worse because it gives a sense of fairness to biased decisions. And the decisions will very likely be biased because they get the data from biased systems.
For example, any sentencing algorithm based on deep learning will give black people harsher penalties because the data they use to train the algorithm comes from a criminal system that is already biased against them. If you leave race out of the training, your algorithm ends up using a bunch of things that are basically proxies for race (address, income, etc.), same thing they used to do (maybe still do?) in bank loans. If you take a lot of care of cleaning your data of any bias, then it loses any statistical significance and is basically worthless.
Actually, I believe AI (based on deep learning) does reason. It’s just that it is not very powerful at the moment. The brain has a lot of very powerful self correcting features that are just not feasible with the current level of technology, that’s what people normally think as reasoning. But for simple things, like reasoning whether a picture is a picture of a cow, AI does a very similar process to what a person says it would do. The reasoning is a bit alien, because the process is different and it involves a lot of low level interactions that humans are not consciously aware of.
Even when AI gets more powerful, its reasoning will remain alien to us. It will be understandable, but alien nonetheless.
On a very unrelated note, AI will not get nearly as powerful as people expect because of hardware limitations. I suspect technology will become a lot more “bio”, with computers made with brains, before we see something resembling human level intelligence.
Thanks for this detailed explanation. Discussing these kind of questions is key if we want to make sure that machine learning benefits society–the many not the few!
There is a lot of discussion atm in the UK on Cambridge Analytics and their impact on the Brexit vote
Given the extent to which our future is being determined by this single (isolated) electoral decision it is important to know how technology might have been used to subvert peoples’ choices.
It’s good to have comments from Tech Experts who understand the mechanics and are willing to engage with more philosophical (offside) questions from those of us with only limited technical understanding. We all get caught up in our own worlds and it is important that tech solutions are not solely the domain of the Techies. So thank you.
Actually, I believe AI (based on deep learning) does reason…
I think you’re redefining the word ‘reasoning’ a bit here. The process by which our brains engage in reasoning is obviously in some ways similar to computer run neural networks, seeing as it involves biological neural networks. The key parts of the definition is in the use of ‘logical’ and ‘steps’. Reasoning is a neural process that is built up, in ways we don’t understand, by multiple separate computational processes in the brain, individually these might behave much like computer based neural networks, but it’s the manner in which they are wired up and interact with each other that allows for complex emergent properties like reasoning to develop, and that is not something we are able to reproduce with AI yet, and once we do, it will only be the totality of the system that can be described as ‘reasoning’, not its constituent parts.
On a very unrelated note, AI will not get nearly as powerful as people expect because of hardware limitations.
I don’t see much evidence for that at the moment, Moore’s law still has some life left in it, and current technology is already many many orders of magnitude faster in terms of raw computational power. Eventually we will hit up against physical limits there, related to fundamental quantum mechanical processes, though who knows what interesting discoveries will be made to allow even further development beyond that. Biological brains have their advantage over CPUs due to the massive level of parallelism - many orders of magnitude greater than the multi-core architectures of a single CPU, but we’re already well on our way to catching up with that, and there are no technical limitations stopping or even slowing our progress in that arena (as the unabated growth in cloud computing has shown, the massively parallel data processing abilities at CERN are another great example). There will also be advances in compilers and processor architecture over the coming years and decades that will help to quicken the pace of development in this too.
I’m not saying there’s no promise to bio/nano-tech in this area, but there’s no evidence yet to suggest it’s a more likely route to AI at this point. Quantum computing is also an interesting avenue, but were still some way away from demonstrating its feasibility (especially in terms of creating a turing complete system, and figuring out exactly how to program it - there is some evidence, not conclusive, that we’ve got somewhere with some less generalised solutions, e.g. D-Wave).
Yes, I guess. Basically my argument is that what we think as human reasoning is a very, very high level computation, but that we also need to reason about low level stuff like correctly labeling a cow as a cow. We do this subconsciously and are not really aware of the steps involved or that it involves any reasoning at all. AI is still at this very low level so we don’t recognize what it does as reasoning, even if it is very similar to what humans do. Also, this is pure speculation on the part of anybody involve, because as far as I know, we don’t really understand how the brain reasons yet
I think the main limitation is efficiency, not computational power. With a large enough computer network it will be possible to approach the computational power of the brain, but the amount of energy and resources required would make it impossibly expensive. The brain is amazing because its is vastly more powerful.than the largest supercomputer network currently available and it only needs the energy available in a loaf of bread.
You are right, and I want to see what it is possible with those new fancy analog chips designed for AI once programmers get good at them. But, it seems to me that there has been a lot of recent advances in bio and people are not thinking about the possibilities. Meat is being grown in labs, and CRISPR seems like a huge advance in genetic engineering. So, I don’t know much about bio, but what if you grow a couple of genetically modified brain cells and interface them with some traditional input/output?
Yes, totally. I was trying to be as vague as possible in my explanation to highlight that.
You’re right, but to develop a superintelligent AI we need only build a single one, so even if it took a sizeable % of the available computational power on the planet to replicate a single human consciousness, that’s likely all we’d need, once we’ve got that sorted the rest should follow fairly naturally (to potentially horrible consequences - I’m currently reading Nick Bostrom’s Superintelligence).
