The recent Apple test on AI models (see Guardian report here) reports that “these models fail to develop generalisable reasoning capabilities”. What does this mean?
Imagine a huge Easter egg hunt. There are a billion rabbit holes, each one a billion miles long, with billions of branches. The egg hunt requires you to find a single chocolate egg. It could be any distance down any hole.
Now, if a computer tried to find the egg the way it tries to solve a complex problem, it would instantly disappear down a randomly selected rabbit hole and vanish for ever. A brain, on the other hand, would be unlikely to order such a futile operation. Recognising the hopelessness of the task, it is more likely to order a visual scan across all the holes to see if an egg, or a clue to one, is visible near the entrance of any of the holes. (If it were a rabbit brain it would probably order a bit of sniffing too.) It’s a small chance, but a much wiser search with a higher chance of success.
The ability to see across all the holes, and not just vanish down one of them, is generalisable reasoning. It means you look at all the possibilities at once, instead of picking one out and vanishing into it. The apple report says that the failure of generalisable reasoning results in the catastrophic failure of AI models to solve complex problems. We shouldn’t be surprised at this. Generalisable reasoning is synchronic. It assesses all the information available at once. Binary computers use a linear process and can’t do it. If quantum computers worked, it might well be what they would do. They don’t work yet, though (and quite possibly never will) so the only machines we know of which are capable of generalisable reasoning are brains. In this respect, rabbits are possibly more intelligent than computers.
The reason why the Apple researchers are concerned, though, is that this kind of reasoning is the most commercially valuable activity humans can perform. It is (and, it appears, will continue to be) trained and enabled by the development of skills in art and creativity, and it leads to innovation, more accurate analyisis and prediction, technological advancement and business success.