You should really go read and subscribe to my buddy Peter’s newsletter. It’s heavy on the AI talk but it’s neither doomerism nor fanboying.
His latest edition, Slop Doctors, is excellent, as usual. I really like this analogy of LLMs making collages.
First, just like a traditional database, when the requested information is not in the database, the LLM cannot produce it. This means that LLMs cannot create. They can make collages, pastiches. They can show you something that you’ve never seen before. But that doesn’t mean it’s a new thing that no one has ever seen before.
Imagine if you had access to every magazine ever printed and a SUPER accurate pair of scissors – this is essentially how LLMs work, though you have to expand the concept of collages to include collages of words, of computer programming code, of songs, of whatever you can have an LLM put together for you.
This is not to say that a collage can’t be art – I have definitely seen many that are truly new works of art. But there is an inherent limit. You must have a scrap of magazine that fits your vision. For artwork, this is typically fine – there are a LOT of magazines out there. It is a HUGE jump from “LLMs make pretty collages” to “LLMs are thinking”. This is why they will never be AGI – there is a limit to the entire concept of LLMs, and I think we’re nearly there. But that’s a conversation for another day.
Anyway, read Peter’s newsletter. It’s way more interesting and coherent than anything I’m putting out.