The more AI is controlled, the less useful it becomes.
- Google is scrambling to fix its AI howlers (see here) but because it has no idea how its machines are working, it is having to play whack-a-mole with a large sledgehammer and wrecking the AI in the process.
- Google has started putting controls and protections into its AI Overview product that was recently rolled out in the USA which predictably resulted in a large number of absurd statements from the AI clearly demonstrating that it has no idea what it is doing.
- These systems operate by matching statistical patterns created during the training process and as such are unable to distinguish between a causal relationship and one that is purely correlated.
- This is why these machines make absurd, untrue and nonsensical statements and remain blissfully unaware that they are doing so.
- The practical upshot of this is that as long as the task for which the machine is trained is both finite (can show the machine every possible example) and static (the rules never change), then these deep learning systems can perform to a superhuman level of ability.
- The problem is that the world is neither finite nor static which means that statistical-based systems have no chance of being able to deal with the task and need a lot of help and plenty of scepticism when it comes to their output.
- Deep learning systems are also plagued with an issue that RFM has dubbed The Black Box Problem which is also referred to as verification in the industry.
- This problem refers to the lack of visibility in terms of how a machine is reaching the conclusion that it does.
- The operator can see the inputs and the outputs but can see nothing of how the machine went from one to the other.
- The model is in effect, a large black box.
- This makes correcting errors extremely difficult and so far, this is being dealt with a heavy-handed approach.
- The operator has no idea why the machine made the error that it did and so instead of a precise correction, the machine is prevented from responding to all requests on a particular subject.
- For example, for yesterday’s newsletter (see here), I asked Copilot to draw a picture of Sam Altman with his fingers in his ears sitting on a timebomb fashioned from sticks of dynamite and an old alarm clock.
- Unfortunately, Copilot interpreted this cartoonish attempt at humour as a request for malicious and violent content and declined to participate.
- I ended up using a boring picture of a pile of broken computers instead.
- The net result is that the more the owners of these models constrain their behaviour to prevent them from misbehaving, the less useful they become and the impact is cumulative.
- As a result of not being able to precisely tell where the machine went wrong in one instance, the model owner has to constrain behaviour much more widely and genuine requests are collateral damage.
- This is why I refer to these fixes as swatting a fly on a bone china plate with a sledgehammer given the impact that they have.
- Furthermore, the nature of the world means that these howlers will keep on appearing because Google is only able to fix the symptoms and do nothing about the problem itself which is the lack of causal understanding inherent to all systems that use deep learning.
- Hence, I continue to think that the best use cases for generative AI will be ones that make use of LLM superpowers which are the use of natural language as an interface and the cataloguing and retrieval of unstructured data.
- Here, there is real money to be made but in terms of superintelligent machines taking over human jobs, we are as far away as we were 10 years ago.
- Hence, expectations continue to run ahead of what is realistically possible meaning that there is going to be a reset.
- I have no idea when this will happen, but I would not want to be anywhere near it when it does.
