LLMs make pretty collages, but they don’t think

You should really go read and subscribe to my buddy Peter’s newsletter. It’s heavy on the AI talk but it’s neither doomerism nor fanboying.

His latest edition, Slop Doctors, is excellent, as usual. I really like this analogy of LLMs making collages.

First, just like a traditional database, when the requested information is not in the database, the LLM cannot produce it. This means that LLMs cannot create. They can make collages, pastiches. They can show you something that you’ve never seen before. But that doesn’t mean it’s a new thing that no one has ever seen before.

Imagine if you had access to every magazine ever printed and a SUPER accurate pair of scissors – this is essentially how LLMs work, though you have to expand the concept of collages to include collages of words, of computer programming code, of songs, of whatever you can have an LLM put together for you.

This is not to say that a collage can’t be art – I have definitely seen many that are truly new works of art. But there is an inherent limit. You must have a scrap of magazine that fits your vision. For artwork, this is typically fine – there are a LOT of magazines out there. It is a HUGE jump from “LLMs make pretty collages” to “LLMs are thinking”. This is why they will never be AGI – there is a limit to the entire concept of LLMs, and I think we’re nearly there. But that’s a conversation for another day.

Anyway, read Peter’s newsletter. It’s way more interesting and coherent than anything I’m putting out.

LLMs aren’t infringing copyright even if you really want them to be

Judge: Just Because AI Trains On Your Publication, Doesn’t Mean It Infringes On Your Copyright

Good post from Techdirt’s Mike Masnick on something I wrote about a while ago (but he has quotes from actual court cases to make HIS point, I’m too lazy for that). He mostly agrees with me, so that’s nice.

Copyright doesn’t cover not liking LLMs

I’ve been thinking a lot about LLMs being trained on content against the will of the content creator. I am very aware of the damage that can be done here, especially to small creators who don’t have a legal budget, and I want to protect their rights, and their opportunity to make a living with their content. But I don’t think, in most cases, these content creators have a right to prohibit their work from being used to train LLMs.

For the sake of argument, there are a few things we’ll ignore. First, clear infringement. If an LLM writes a full-length Hunger Games sequel with the same characters, in the same universe – this is clearly already covered by copyright, this is clearly infringement. Important but intellectually boring. Second, electricity needed to power the servers housing the LLMs. Also important, also boring from an intellectual property perspective.

Also, it’s not AI. I like “spicy autocomplete” but whatever you call it, it’s not “intelligence”. It’s simply making guesses based on all the content it has ingested. It can’t make new connections. This is GOOD – we’ve all seen Terminator and no one wants to live in that universe.

We will also assume that the content has been obtained legally. Unauthorized content is a problem but also uninteresting in this context. People getting content through unauthorized means was a problem before LLMs and will be a problem going forward, even if LLMs disappeared today.

So take an anecdote. Let’s say I am a huge fan of Stephen King. I can read all his books (even the ones my friend’s mom swore were written by his wife). This will surely influence my writing style (and in fact it has, because I AM a fan of Stephen King, and have read dozens of his books. It would influence my fiction even more if I got around to writing any with any sort of frequency). This is clearly not any sort of copyright infringement. So, training your LLM on legally obtained copyrighted content is ALSO not copyright infringement.

Next, with my newly earned writing chops, I can write a 1,500 page sequel to The Stand. If I’m good enough, it will sound a bit like he wrote it. If I keep this on my laptop and only read it to pat myself on the back, this is completely legal and does not infringe on his copyright in any way.

Now I try to sell The Stand II – Standoff under my new pen name, Steven Kimg. This is VERY CLEARLY copyright infringement (and remains so even if I’m a bit more subtle with my marketing). Enforcement of these laws is hard, but it’s not impossible. I’m in favor of better enforcement of these laws to protect content creators, but that has little to do with LLMs. Ask any author how many infringing copies of their book were available on Amazon 3 years ago, before LLMs were mainstream.

What if my friend, who is ALSO a King superfan, pays me to write the book? He plans to keep it for himself and not show it to anyone else. For someone like Stephen King, this is too small to matter. He would probably be annoyed at me if he ever found out, but I can’t imagine he’d bother calling his lawyer. A small content creator might be angry, and justifiably so, but showing real damage would be difficult even though I think this is also copyright infringement.

But what LLMs are doing is largely not the same as any of the above. They are reading all of Stephen King, and all of Suzanne Collins, all of Tumblr and Reddit, and anything else they can get their “hands” on. This is literally exactly what humans do to develop their own craft, and I don’t think the volume at which the LLM may do this as opposed to the volume at which a human does it makes any difference to how the law applies. If I read a book and it influences my art, that is not copyright infringement. If I read 100 and they influence my art, still not infringement. 1,000? Still no. 1,000,000? Still no, though this would be a difficult feat for a human.

The problem that isn’t well covered by existing law is when the artist doesn’t want their work used to train these LLMs. I don’t think that is a protected right. It’s like when a politician licenses a song from the label and plays it at a rally. The artist gets mad because they disagree with the politics. The politician may get bad publicity for this, but they are 100% within their legal rights to continue using the song (again, assuming it’s legally licensed, because if it’s not then it’s not interesting to discuss, it’s just boring infringement). Another example – the creators of The Boys have complained that many people who watch the show come away thinking Homelander is the hero. He is quite obviously a deranged sociopath, though I absolutely love the character. But this is a similar case of authorized users of your content using it for something you hate (promoting sociopathic superheroes).

If we want to prevent this, we need new laws. Copyright is a giant hammer and modern content creation and sharing requires a much more versatile tool. Creative Commons tried to provide this and it caught on in some circles but never got the critical mass from big companies, probably because they’re just fine with the giant hammer – they have the legal resources to back it up and don’t much care about the collateral damage. I’m not optimistic we’ll resolve this – the Venn Diagram of those with the desire to change and the power to change is probably two separate circles. But maybe if we think about it this way, we can save some whining.

I feel like we’ve heard this before but it still sucks

“The Small Press World is About to Fall Apart.” On the Collapse of Small Press Distribution

You can go back through the archives of this blog, hardly an authority on anything (unless AI content farming has really killed off every other blog and then maybe we are), and you can see so many instances of “This is going to kill [some aspect of the publishing world]” and largely it just hasn’t happened.

This one does sound bad. SPD, one of the last small distributors, is going under. They’re doing it quickly, and so far leaving some clients unpaid. It’s already incredibly hard as a small author to get your book noticed by the mainstream, and if nothing else steps in to fill the void left by SPD’s demise, it’s going to be a whole lot harder. People are very stubbornly clinging to their paper books, and while I don’t entirely blame them, it’s just not sustainable going forward.

I have a fair bit of faith in authors and their ability to pivot, but we keep making it harder on them and that’s no way to encourage creation.

For example, Kameron Hurley is one author offering a monthly Patreon subscription where you get exclusive stuff. It’s cool. I absolutely love her universe where some people can inhabit corpses. It’s a really well-developed universe that she has sadly (to me, at least) not written nearly enough in. But plugging an author I like is not the point (though it’s a bonus). This is all extra work. It used to be you could just be an author and your agent would work and get your books in front of people. Maybe that worked and maybe it didn’t, but that was about it. Now authors have way more opportunity but also way more hats they have to wear.

We’ve been talking for a decade at least about alternate paths to success for authors and they mostly haven’t materialized. Maybe authors should try using the electrical output of a mid-sized country to write a book and maybe Silicon Valley would take notice and throw some venture capital at them.

Never thought I’d see the day – welcome to the Public Domain, Mickey

I’m a few months late on this as I was still on blogging hiatus when it actually happened. but the earliest version of Mickey Mouse (The one that Disney very likely stole from another artist) is finally in the Public Domain. I guess Disney decided the lobbying dollars would be more valuable somewhere else and they didn’t get Congress to retroactively extend copyright again.

It’s all the rage lately to be a Constitutional Originalist, but what that really means is you do it when convenient. Otherwise the Supreme Court would obviously have to overturn the laws retroactively extending copyright. Copyright was meant to “promote the Progress of Science and useful Arts” – it literally says that in the Constitution. It was supposed to give people incentive to create things that others would find useful or beautiful. Adding years to an existing copyright can’t do that – the covered work has already been created.

You can argue that extending copyright on future works would promote the progress, but that’s a different argument. It’s still wrong, but that’s an argument for another day.

Extending copyright on existing works is simply a handout to someone who already took your deal.

This is all kind of silly at this point anyway – Mickey Mouse has evolved quite a bit since 1928, with most versions are still protected, AND most of what people wanted to do with the freed Mickey were already permitted under fair use. Still, I’m happy to see this day, as I never thought I would.

Activitypub testing, mostly

I’ve just installed the Activitypub plugin and so far I can’t find it from Mastodon, and I assume this is because I haven’t posted anything to the blog yet since I installed the plugin.

I’m going to try to get the blog going again. I used to enjoy doing this. The ebook market has not evolved nearly as much as you would have thought in all these years, but at least it’s not all Amazon anymore. There aren’t a ton of cool new business models built up around ebooks. I know my kids read ebooks sometimes on their phones, but are more likely to read a paper book. Whether this is the cause of the lack of business models around ebooks, or the effect, I don’t really know.

One book does not a reversal of policy make

I don’t share the optimism of Teleread and Kindle Nation, but it appears that Amazon has not entirely deserted free ebooks.

They do, however, remain committed to controlling virtually every aspect of the Kindle that they’ve leased you, which is not terribly consumer-friendly.

Still, at least it’s not as bad as it originally seemed.

Free ebooks back at Amazon- John Lutz Urge to Kill | TeleRead: Bring the E-Books Home.