That assumption isn't true. Socialists aren't born that way, most come out of the status quo ideology of liberalism. By abandoning all liberals with blanket statements, we'd simply self-fulfill that prophecy. Even US libertarian militias, a peak of liberalist ideology, have sometimes sided with antifascists over fascists (see: Redneck Revolt lines of affiliation with American Pit Vipers).

You're referencing a real trend, and there's a kernel of truth behind it, however it's harmful to the socialist movement to assume that as a universal inevitability.