Programmer Humor

Yeah I read one of the papers that talked about this. Essentially putting AGI data into a training set will pollute it, and cause it to just fall apart. Most LLMs especially are going to be a ton of fun as there were absolutely no rules about what to do, and bots and spammers immediately used it everywhere on the internet. And the only solution is to.... write a model to detect it. Which then they'll make models that bypass that, and there will just be no way to keep the dataset clean.

The hype of AI is warranted - but also way overblown. Hype from actual developers and seeing what it can do when it's tasked with doing something appropriate? Blown away. Just honestly blown away. However hearing what businesses want to do with it, the crazy shit like "We'll fire everyone and just let AI do it!" Impossible. At least with the current generation of models. Those people remind me of the crypto bros saying it's going to revolutionize everything. It might, but you need to actually understand the tech and it's limitations first.

It's definitely a qualitative shift. I suspect most of the fundamental maths of neural network matrices won't need to change, because they are enough to emulate the lower level functions of our brains. We have dedicated parts of our brain for image recognition, face recognition, language interpretation, and so on, very analogous to the way individual NNs do those same functions. We got this far with biomimicry, and it's fascinating to me that biomimicry on the micro level is naturally turning into biomimicry on a larger scale. It seems reasonable to believe that process will continue.

Perhaps some subtle tuning of those matrices is needed to really replicate a mind, but I suspect the actual leap will require first of all a massive increase in raw computation, as well as some new insight into how to arrange all of those subsystems within a larger structure.

What I find interesting is the question of whether AI can actually fully replace a person in a job without crossing that threshold and becoming AGI, and I genuinely don't think it can. Sure it'll be able to automate some very limited tasks, but without the capacity to understand meaning it can't ever do real problem solving. I think past that point it has to be considered a person with all of the ethical implications that has, and I think tech bros intentionally avoid acknowledging that, because that would scare investors.

I'm sure it would be pretty simple to put a simple code in the pixels of the image, could probably be done with offset of alpha channel or whatever, using relative offsets or something like that. I might be dumb but fingerprinting the actual image should be relatively quick forward and an algorithm could be used to detect it, of course it would potentially be damaged by bad encoding or image manipulation that changes the entire image. but most people are just going to be copy and pasting and any sort of error correction and duplication of the code would preserve most of the fingerprint.

I'm a dumb though and I'm sure there is someone smarter than me who actually does this sort of thing who will read this and either get angry at the audacity or laugh at the incompetence.

I see this a lot, but do you really think the big players haven't backed up the pre-22 datasets? Also, synthetic (LLM generated) data is routinely used in fine tuning to good effect, it's likely that architectures exist that can happily do primary training on synthetic as well.

AIs can be trained to detect AI generated images, so then the race is only whether the AI produced images get better faster than the detector can keep up or not.
More likely as the technology evolves AIs, like a human, will just train real-time-ish from video taken from it's camera eyeballs.
...and then, of course, it will KILL ALL HUMANS.