You could restructure your network, but it's probably not necessary. My phone is always behind NAT on the secondary router's wi-fi. I got it working by:
- Reserving a static IP address for my phone in the secondary router's DHCP server.
- Forwarding that router's incoming ports 1714-1764 to the phone's IP address.
- Allowing outgoing traffic from that router's network to ports 1714-1764 on my workstation's IP address (on the primary network).
- Adding a device (my workstation) by IP address in my phone's KDE Connect app.