Artificial Intelligence – Missing Link

The link is still missing.  

• Meta and OpenAI are claiming that their next models will be capable of reasoning which, if true, will be a key step forward in the quest to achieve super-intelligent machines, but I suspect that this, just like the models that have gone before, will offer a simulation of reasoning and nothing more.
• RFM Research has argued for 6 years that the main limitation of all AI systems that are based on deep learning is that they have no causal understanding of what they are doing.
• Instead, they are highly sophisticated pattern recognition systems that can identify correlation but unable to decide whether a relationship is causal or simply correlated.
• It is this weakness that causes the failures in the most simple neural networks all the way up to the sophisticated large language models (LLMs).
• The ability to reason requires a causal understanding of the task at hand and consequently, if it can be implemented in deep learning systems, this would represent a big step forward in the capability of these machines.
• It is ironic that the same actors who, when launching the current generation of LLMs, claimed that they could reason, now admit that they can’t when talking about how the next generation may have this capability.
• The Financial Times (see here) has put together a piece that makes this claim but picking apart what these actors have actually said, one can see how uncertain this outcome is.
  • Yan LeCunn, Chief Scientist, Meta Platforms: “I think over time . . . we’ll see the models go toward longer, kind of more complex tasks, …and that implicitly requires the improvement in their ability to reason.” (FT).
  • Brad Lightcap, COO OpenAI: “We’re going to start to see AI that can take on more complex tasks in a more sophisticated way … I think we’re just starting to scratch the surface on the ability that these models have to reason.” (FT).
  • Joelle Pineau, VP AI Research, Meta Platforms: “We are hard at work in figuring out how to get these models not just to talk, but actually to reason, to plan  . . . to have memory,” (FT).
• Meta’s commentary is far more realistic than OpenAI’s as Meta expresses an aspiration whereas OpenAI seems to be implying that it thinks that it can implement reasoning in a purely statistical-based system.
• I have serious doubts that this is possible as the only way I know of achieving reasoning in computer systems is to use the rules-based software that has been around for years.
• The reason why a cheap pocket calculator is better at maths than a multi-billion dollar LLM is because it is programmed with the rules of maths whereas the LLM uses statistics and probability to achieve its results.
• In short, software can reason but it can’t learn whereas deep learning can learn but it can’t reason.
• Consequently, it stands to reason that if one were to combine software with an LLM, this would represent an effective and efficient method of implementing reasoning in these systems.
• This is referred to as neuro-symbolic AI (symbolic is a fancy term for software) and was a relatively active area of AI research 4 years ago (see here) but it has been pretty quiet since all the focus moved to LLMs.
• Unless OpenAI is referring to using software in its LLMs to provide the reasoning, then I am certain that GPT-5, LlaMa 3 and so on will be as incapable of reasoning as all of their predecessors.
• These systems are very good at simulating reasoning but when they are given real empirical reasoning tests, they fail and fail convincingly.
• Hence, despite the chatter and the hyperbole, I continue to think that we remain as far away from super-intelligent machines and artificial general intelligence as we have ever been.
• This means that hundreds of millions of jobs are safe for the foreseeable future and that the robots are not coming to kill us yet.
• It also means that we are in an AI bubble and at some point, everyone will realise what this new branch of AI can realistically do and what it is just noise.
• Generative AI is capable of many things especially its use of natural language as a user interface and the categorisation and storage of unstructured data in a useful and easy-to-access way.
• This has real use cases, but it is not the genesis of superintelligent machines that send the human race into retirement as the market seems to think.
• The causal link is still missing.