I just stumbled upon the solution for the specific problem I was facing 🎉.

Use @media (inverted-colors) does allow you to detect if a user inverted their colours!