This is extremely cool.

Because of the federated nature of Lemmy many instances might be scanning the same images. I wonder if there might be some way to pool resources that if one instance has already scanned an image some hash of it can be used to identify it and the whole AI model doesn't need to be rerun.

Still the issue of how do you trust the cache but maybe there's some way for a trusted entity to maintain this list?

As a test, I ran this on a very early backup of lemm.ee images from when we had very little federation and very little uploads, and unfortunately it is finding a whole bunch of false positives. Just some examples it flagged as CSAM:

• Calvin and Hobbes comic
• The default Lemmy logo
• Some random user's avatar, which is just a digital drawing of a person's face
• a Pikachu image

Do you think the parameters of the script should be tuned? I'm happy to test it further on my backup, as I am reasonably certain that it doesn't contain any actual CSAM

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:

4 acronyms in this thread; the most compressed thread commented on today has 6 acronyms.

[Thread #88 for this sub, first seen 28th Aug 2023, 22:25] [FAQ] [Full list] [Contact] [Source code]

How do you even safely test scripts/tools like this 😵‍💫

I don't host a server myself, but can this tool identify the users who posted the images and create a report with their IP addresses?

This could help identify who spreads that content and it can be used to notify authorities. No?

Thank you for helping make the fediverse a better place.

Worth noting you seem to be missing dependencies in requirements.txt notably unidecode and strenum

Also that this only works with GPU acceleration on NVidia (maybe, I messed around with trying to get it to work with AMD ROCm instead of CUDA but didn't get it running)

Don't have a GPU on my server. How is performance on the CPU ?

Hi db0, if I could make an additional suggestion.

Add detection of additional content appended or attached to media files. Pict-rs does not reprocess all media types on upload and it's not hard to attach an entire .zip file or other media within an image (https://wiki.linuxquestions.org/wiki/Embed_a_zip_file_into_an_image)

Not well versed in the field, but understand that large tech companies which host user-generated content match the hashes of uploaded content against a list of known bad hashes as part of their strategy to detect and tackle such content.

Could it be possible to adopt a strategy like that as a first-pass to improve detection, and reduce the compute load associated with running every file through an AI model?

based db0 releasing great tools and maintaining a great community

Opened a relevant issue: https://github.com/LemmyNet/lemmy/issues/3920

Hey db0 thanks for putting in extra effort to help the community (as you have multiple times) when big issues like this crop up on Lemmy.

Despite being a pressing issue this is one that people also are a little reluctant to help solve because of fear of getting in trouble themselves. (How can a server admin develop a method to detect and remove/prevent CSAM distribution without accessing known examples which is extremely illegal?)

Another time being the botspam wave where you developed Overseer in response very quickly. I'm hoping here too devs will join you to work out how to best implement the changes into Lemmy to combat this problem.

Thanks for releasing this. After doing a --dry_run can the flagged files then be removed without re-analysing all images?

This is awesome. Thank you for making it.

Any thoughts about using this as a middleware between nginx and Lemmy for all image uploads?

Edit: I guess that wouldn't work for external images - unless it also ran for all outgoing requests from pict-rs.. I think the easiest way to integrate this with pict-rs would be through some upstream changes that would allow pict-rs itself to call this code on every image.