Key points off the top of my head:

• Both Signal and Matrix have solid cryptography. (In fact, the Matrix Megolm protocol is built upon Signal's Olm protocol.)
• Signal's client is open-source, but not its server. Matrix clients and servers are open-source. (Note that open server code is less important than open client code when end-to-end encryption is used.)
• Both have had security audits, and presumably will have more in the future.
• Both have cross-platform support. Matrix has a greater variety of clients, and last time I checked, the desktop clients were better than Signal's.
• Signal prides itself on minimizing metadata exposure. Matrix started later, and hasn't yet moved all metadata (e.g. reactions, group membership?) to an encrypted channel. (Note that metadata protection can't ever be complete for either one of them, because an attacker that can gain access to the servers could just as well gain access to the routers, and correlate packets at the network level.)
• Signal is a centralized service (hosted by Amazon last time I checked), meaning all traffic flows through one place and can be shut down or monitored there. Matrix is federated, much like Lemmy.
• Signal does not offer self-hosting. Matrix does. (One might do this to be sure they always have a copy of their past messages, or to be independent of public server outages, or to be the sole keeper of data and metadata in conversations between local users.)
• Signal requires a phone number, which makes having an anonymous account challenging for many people. Matrix doesn't require anything, but some Matrix servers require an email address for account recovery purposes.
• Signal apparently integrates cryptocurrency payments. Matrix does not.

The most obvious security vulnerability that I can think of is that the person you are communicating with (or, conceivably, oneself, as well) is using an insecure/compromised application that may be leaking information.

Indeed, end-to-end encryption is unable to protect the endpoints. That's true of both messengers.

Since you brought it up, note that smartphones generally run system-level software that is controlled by the OS maker, meaning that Google or Apple have the ability to monitor those endpoints. Signal's approach to addressing this is to offer a Google-free build of their app, if you can find it, buried on their web site and with a scary-looking warning against using it. Matrix publishes their apps on F-Droid, so there's an easy Google-free option. Of course, most of the people using either messenger will have downloaded from the big app stores, so even if your phone and app are Google-free, it's likely that most of your contacts' phones are not. I believe Matrix has an advantage here, simply because their Google-free builds are easier to find and keep updated, and are therefore probably in use by more people.