No Stupid Questions

For images, yes. Most notable is probably Adobe. Their AI, which powers photoshop's generative fill among other things, is trained on public domain and licensed works.

For text, there's nothing similar. LLMs get better the more data you have. So, the less training data you use, the less useful they are. I think there are 1 or a few small models for research purposes, but it really doesn't get you there.

Of course, such open source projects are tricky. When you take these extreme, maximalist views of (intellectual) property, then giving stuff away for free isn't the obvious first step.

It's not an issue to me, and is completely befuddling to begin with. Training an AI on copyrighted material doesn't mean the AI violates that material when it generates new artwork. AI models don't contain a copy of all the works they were trained on - which could be petabytes of data. They reduce what they learned to math algorithms and use those algorithms to generate new stuff.

Humans work much the same way. We are all exposed to copyrighted material all the time, and when we create new artwork a lot of the ideas churning inside our heads originate from other people's works. When a human artist draws a mouse man smiling and whistling a tune, for some reason it's not considered a copyright violation as long as it doesn't strictly resemble mickey mouse. But when an AI generates a mouse man smiling and whistling a tune? Suddenly the anti-AI crowd points at it and screams about it violating Disney IP.

It's not an issue. It never was. AI training is a strawman argument manufactured by the anti-AI crowd to justify their hatred of AI. If you created an AI trained on public domain stuff, they would still hate it. They would just clutch at some other reason.

As I understand it, there are many many such models. Especially those made for academic use. Some common training corpus's are listed here: https://www.tensorflow.org/datasets

Examples include wikipedia edits and discussions, and open source scientific articles.

Almost all research models are going to be trained on stuff like this. Many of them have demos, open code, and local installation instructions. They generally don't have a marketing budget. Some of the models listed here certainly qualify: https://github.com/eugeneyan/open-llms?tab=readme-ov-file

Both of these are lists that are not so difficult to get on; so I imagine some of these have trouble with falsification or mislabeling, as you point out. But there's little reason for people to do so (beyond improving a papers results I guess?).

Art generation seems to have had a harder time, but there are stable diffusion equivalents that used only CC work. A few minutes of search found: Common Canvas, claims to have been competitive.

I'd say the biggest issue with generative AI is the energy use and the fact that it's increasing the rate at which we're destroying the climate and our planet.

https://allenai.org/

I worked at a major tech company and their attitude was 'if this becomes popular enough, and copyright is an issue, we'll just pay artists to produce training data en masse'.

For text generation the result would be almost useless since most public domain works are very old. For images, you could train with video feeds maybe.

using human-made works where the original authors didn't consent to or even know that their work is being used to train the AI. Are there any initiatives to address this issue?

I don't think so.

Training is expensive, and they all want to make money. Most of them want to make insane amounts of money.

Therefore they don't care if it's illegal.

Free and Nonprofit usually don't start with huge piles of play money. OpenAI was the exception once before it got so thoroughly corrupted.

Hey PubDomainLLM tell me something that only exists in that proprietary dataset? “I’m sorry, you’ve caught me lackin’”

You would want your LLM to be trained on as comprehensive a dataset as you can. But I would suggest we should be coming up with better ways to license proprietary works for uses like this instead of walling it up for the cable tv of proprietary knowledge gardens.

I agree with you partially in principle but not in practice.

Ultimately we want as smart LLMs as we can, just compare the best models with the mediocre ones, or use them all day long, there is a vast difference.