Tags, tags, tags.
My photos are stored by date ("yyyy/mm/dd - Main description", e.g. "2018/04/19 - Lofoten holiday"), but that is by far not enough to find anything meaningful at any time.
Meticulous tagging with a well-structured taxonomy is the only real choice here, in my book. AI tools, like are integrated into immich and other tools, help building that library. But for what I do with photos, how I think about photo gategories, AI can only go so far.
Tagging appropriate subject matter (family, friends, still lives, architecture), surroundings (location, as in "Lofoten", but also location, as in "at the beach"; indoor/outdoor; nature; event; ...), theme (light painting, black-and-white, nighttime, high fashion, random funny snapshot, ...) and even species (I photograph a lot in zoos) is necessary.
With that kind of thing you then can find "that funny photo of Uncle Roger at the beach, when he slipped on the stray dog toy!" in the 4TiB of photos.
And that's, unfortunately, work.
(Also, I am harsh in culling my RAW files. Ofc precious memories stay even if the photo isn't perfect, but if I go out on photo walks, anything that is blurry/unsharp or badly exposed has the original RAW deleted as well. Or selecting the best candidate from chimping, I don't need the other ones anymore. But that, too, is a workflow problem.)
As the saying goes: That you're paranoid does not mean they're not out to get you ... ๐
I have those battle scars too. :(