This happens to me on the PC with my xbox gamepad as well. It started recently, since the patch before the ps5 launch, I feel.
The best quick solution on gamepad I found is to short press 'up' on the dpad to enter jumping which cancels the targeting.
Btw, holding up un-/equips the torch, holding down let's you enter stealth (for the whole party, I think) and tapping down let's you immediately examine the targeted object/creature ✌️
It has no backlighting, sadly... right?