Anthropic & Open AI – Mission impossible

Breakaway start-up has an impossible mission.

• Anthropic is a new breakaway start-up from Open AI but its approach remains pretty much the same meaning that its focus on “reliable, interpretable and steerable AI” is going to be very hard to achieve.
•  Anthropic has also just raised $124m at a valuation of $845m which is pretty incredible as there does not seem to any product in the pipeline or any mention of a commercial concern.
• In fact, this may have been the reason for Anthropic’s creation as the $1bn raised from Microsoft pushed Open AI in a more commercial direction.
• Furthermore, I suspect that when someone hands out $124m at a valuation of $845m with no real requirement to produce anything, no one in their right mind is going to say no.
• It is notable that the investor list is headed by billionaire founders with money to burn as opposed to VCs who have to make a return for their investors.
• Hence, I suspect that the funders of this venture view it as philanthropy rather than investment.
• Hence, It may well have been “free money” that caused the split rather than any real disagreement over the direction of research or fundamental approach.
• As far as I can tell, Anthropic is doing pretty much the same as Open AI meaning that it will come up against all of the same limitations.
• Put very simply. Open AI (and now Anthropic) are proponents of what RFM research refers to as the brute force approach to solving AI.
• The idea here is that with enough compute power and enough data, the limitations of AI can be overcome and general intelligence can be achieved.
• General intelligence is the ability to take what has been learned from one task and apply it to another.
• The problem is that in order to do this, the system needs to have causal understanding of the task at hand such that it can adjust to any changes in the environment.
• Deep learning systems have been demonstrated time and again to badly fail even the most simple tests of this ability which is because at their heart they are no more than data pattern matching systems.
• Open AI has published numerous papers on this subject but after careful analysis, RFM has concluded (see here) that while these represent interesting advances, Open AI is no closer to solving the generalisation of AI problem than when it was founded.
• However, Anthropic is taking the challenge to another level by also targeting reliability and the ability to interpret how the AI is working.
• In any critical system (e.g. autonomous driving) the system needs to explainable in that one can easily tell how it works and why it does what it does.
• Deep Learning by its very nature is a black box in that one can see the inputs and the outputs but not how the outputs are derived from the inputs.
• This raises another big problem for Anthropic which on top of being tasked with creating general AI (like Open AI) its task is also to make it “interpretable” meaning that Anthropic needs to be able to explain how its systems do what they do.
• Using deep learning, this is going to be extremely difficult leading me to conclude that the tasks that Anthropic has set for itself will not be achieved using its current methods.
• RFM research still concludes that the most promising approach will be one that combines traditional software and deep learning, but this remains an area of academic research rather than any commercial product.
• Hence, the pace of development is likely to continue to slow inexorably leading to a period of disappointment and disillusionment otherwise known as an AI winter.
• The cold forecast remains in place.