Research Publication – Reality Bytes: AI – Old school

August 17th 2020: RFM updates its coverage of AI with the publication of Reality Bytes – AI: Old school.  

RFM subcribers will receive their own copy by email / secure download.

The pandemic has pushed AI out of the limelight, but a 3^rd AI winter remains inevitable. This is because, despite a deluge of research, deep learning has not materially advanced. However, lottery tickets do promise to move AI at the edge meaningfully forward. Alternative approaches include throwing infinite data and compute at the problem and a hybrid approach marrying deep learning with software. Of all of these approaches, the hybrid approach has the most promise and is especially interesting for autonomous driving.  

• COVID-19. AI is created in the cloud meaning that its makers can just as easily work on it from home resulting in little if any, disruption. However, the pandemic represents a seismic shift in many real-world datasets meaning that widespread retraining is likely to be needed. The availability of enough data to retrain for COVID-19’s changes is also an issue.
• AI Winter. Very little progress has been made in real-world applications which, combined with assistants being as dumb as ever, makes deep learning’s shortcomings even more visible. RFM still sees a lot of disappointment coming which will trigger falling investment and valuations. Winter is coming although efforts to mitigate its impact are swinging into gear.
• The nuclear fusion of AI is the creation of lottery tickets. These are trained neural networks whose size has been cut back by as much as 90% but still function almost perfectly. However, they are prohibitively expensive to create meaning that (like nuclear fusion) they consume far more than they give back. This avenue of research could make the deployment of AI at the edge much cheaper with far less storage and compute being required. However, it does not meaningfully address some of the fundamental shortcomings in RFMs opinion.
• Brute force: is the idea that with enough compute power and enough data, deep learning will achieve everything that is expected of it. OpenAI is the highest-profile proponent of this approach, but RFM finds that its research has too many caveats and does not live up to the hype. Its latest release (GPT-3) is another example of the infinite monkey theorem.
• Old School. The most promising branch of AI is its oldest (symbolic AI (or in layman’s terms: regular software)). RFM find that the combination of deep learning networks and regular software has the potential to mitigate some of the limitations that keep deep learning from really pushing AI forward. Research has demonstrated the potential to produce real progress against two of the most vexing problems plaguing AI: (data quantity and transfer learning).
• Autonomous driving is one of the best potential use cases for this hybrid approach. This is because it is a fairly uniform task across the world where only the dataset changes. RFM thinks that structuring the cognitive piece in symbolic AI with multiple neural nets being used for specific vision tasks is how this technology will be commercially delivered.