It doesn’t have “memory” of what it has generated previously, other than the current conversation. The answer you get from it won’t be much better than random guessing.

For my understanding artistic works get copyright from the moment of their creation. This would allow one to pick battles based on how lucrative they may potentially be.

In the US, you need your copyright to be registered in order to file an infringement suit or be granted statutory damages. This must be done prior to the infringement, so they wouldn’t be able to pick and choose which to register after the fact. The fact that (unregistered) copyright arises from the moment of creation is true, but not particularly useful here.

You dont really need art museum of babel for this but you just tons of different works that may contain unique characters, structures or objects similar to what someone might be able to imagine or has already imagined.

Copyright is not the same as patents or trademarks; someone coincidentally creating something very similar or even an exact replica of your work is not infringement.

If whether you copied from their work or independently made similar choices is under question - then close similarity of the works could skew the balance of probabilities. However, the courts will be able to see that coincidental similarity is far more likely if a colossal number of images have been registered.

You may draw fan art with disney characters but its actually illegal to sell said art work without Disneys aproval until copyright expires.

It’s still copyright infringement even if you publish it non-commercially, but a Fair Use defense would likely hold up.

To file an infringement suit they’d need to have paid registration for each work which, even for the exorbitantly rich, wouldn’t be remotely feasible for all logical arrangements of words/images. There’s probably not even enough space in the Universe or time until its heat death to generate and store all such images.

Even if they did, copyright doesn’t protect against against independently created works that happen to be similar or even identical - so they wouldn’t be exhausting some limited set of possible works by doing so.

they are looking for specific ones that are in a database

They could be looking for any images without your knowing - there’s no guarantee that those images came from a CSAM database.

Its not like they can create a hash for a guy letting his dog on a horse

They could trivially create a hash for a picture of a guy letting his dog on a horse (which would also include other very similar images).

I didn’t necessarily mean to claim that they can scan for a concept lacking a fixed image, if that’s what you’re saying. That would theoretically be possible with enough hashes, but impractical.

And how was it privacy compromising?

1. Anything could be added to the hashes with the user having no way to know what’s being searched for beyond “trust us”. This could be partially alleviated if, for example, the hash had to be signed by organizations in a combination of states that’d make it difficult to push through hashes for anything other actual CSAM

2. Adversarial examples to intentionally set off the filter were demonstrated to be possible. Apple made it clear that there are types of content they’d be legally obligated to report once they became aware of, and it’d be well within a government agency’s capabilities to honeypot, say initially, terrorist recruitment material

3. Coincidental false positives are also entirely possible (ImageNet had some naturally occuring clashes) and can result in their employees seeing your sensitive photographs

4. The user’s device acting against the user cements other user-hostile and privacy-hostile behavior. “People could circumvent the CSAM scan” would be given as another reason against right to repair and ability to see/modify the software your own device is running

5. Tech companies erode privacy by flip-flopping between “sure we’re giving ourselves abusable power, but we’ll stand up to governments pressuring us to expand this” and then “well what were we supposed to do, leave the market?” when they inevitably concede