I think it’s also worth pointing out that like airplanes and birds, they’re not even trying to work the same way. Jaded found this nice diagram of GPT4:
That is a hell of a lot of neurons, but look at the arrows – they all point the same way. That’s why there’s a training phase to set the weights and then a separate application phase that uses them. So we know it is fitting a hypersurface to the input vectors, even if the actual hypersurface is going to be far too convoluted to describe otherwise.
For comparison, a nematode worm has only 300 or so neurons, but they actually feed back into each other. That adds an entirely different sort of complexity to the network. So many people keep talking about LLMs as if they were on the verge of some kind of consciousness, but they’re actually much less like it than the microscopic worm is, which can’t learn much but still learns as it goes.
