Technology

So the standard approach to this is so-called "perceptual hashing." Effectively, using cryptographic hashes (sha256, etc.) doesn't really work well in this case. Given a piece of illegal content, that content is likely to still be just as illegal with a single pixel changed -- however, it'll have a completely different cryptographic hash. So instead, a hash function that determines how "similar-looking" two images are, ignoring things like dimensions, color palette, JPEG compression artifacts, etc. This is obviously way fuzzier, and is prone to both false positives and negatives.

Because all this is inherently kinda fuzzy, the exact database of hashes is usually "secret sauce" if you will. If it were public, it would be super easy to circumvent. As an example, given an illegal image:

1. Is the image's hash in the DB?
2. No? All done, you can post it with impunity.
3. Yes? Change one random pixel, GOTO 1.

As a result even "public" databases are distributed with NDAs etc. This obviously does not jive well with an open source, federated network like Mastodon, and I have my doubts as to how willing the relevant agencies would be to give their databases to every rando with $5 to spin up a Pleroma instance on a VPS. A public DB might help in some cases, but unfortunately more illegal content is produced every day, and so it would be extremely hard to keep up with the bad actors.