They're "invasive" because humans brought them into new ecosystems where they shouldn't be in the first place. Plants (and animals) just don't just go around invading other ecosystems afaik.

Also, elephants taking down trees, they're (mostly) supposed to do that. It plays a vital role in the ecosystem too. Elephants taking down large trees makes it easier for smaller plants to grow, making the ecosystem more diverse. Also these trees and the elephants have co-evolved for centuries, so I they know how to live with eachother and can probably do that for years to come.

Also, that's besides the point. Other animals aren't as conscious as us. We are not just following instincts the same way as them. We know stuff we're doing is destroying the planet, yet we continue because $$$