RTS means real time strategy, it's stuff like warcraft, starcraft, age of empires etc. X-COM is turn based, so not real time.

If you are searching for turn based tactical games (I don't really play RTS), yeah, Mario + Rabbids is very reminiscent of X-COM, only with an emphasis on movement combos (using other characters/enemies to boost your jumps, that kind of things). It does have multiplayer maps, though I have not tried them myself. Main campaign is single player.

In that move combo aspect it looks quite a bit like Chroma Squad, a tactical game with a cast of super sentai/power rangers-like actors who do combo stunts as they move. It's pretty fun, though I have not tried the switch version and I think it's single player only.

Fire emblem has two episodes on switch, three houses and engage. I have played a lot of three houses. It's not for everyone, it's a very long game with loads of dialogue, many characters, a lot of schedule planning etc. Battles are turn based, though in Fire Emblem fashion more about tricking the (basic) AI to break against your high defence "wall" characters while protecting your glass cannons characters. Difficulty being there is a lot of those (admittedly dumb) enemies with various strengths and weaknesses. It's fun but again, single player only. I haven't played Engage, because honestly, Its character design is terrible and it looks quite silly.

Triangle Strategy is another turn based tactical RPG with an heavy emphasis on story. Lots and lots of dialogue, to the point you may ask yourself when the strategy starts when you begin. It has branching story paths, with a rather unique voting mechanic where you have to convince your people to choose the path you want to take. Battle maps are typically less flat than fire emblem, and turn order is determined by each unit's speed instead of being player turn/enemy turn. Again, single player.

Into the Breach is turn based strategy on a very small scale (each battle is 5 turns on a 10x10 map). It's almost more of an open-ended puzzle than a strategy game, requiring you to use your squad effectively to defend key locations against waves of bugs with different abilities. It's very good.

Wargroove is Advance Wars inspired, so instead of unique characters like Fire Emblem etc, you play one commander with a special ability, and armies of nameless soldier units that you recruit in cities you control. A remake of Advance Wars 1 and 2 is on switch too, I have not played it. Not a fan of this type of game personally, but they have their fans. Those are multiplayer, kind of like chess with more rules.

Now for something turn-based and strategic but completely different, Civilization 6 is on the switch, though I don't know how well it works on it. It's Civilization, so long games on random maps where you found your cities, find resources, develop technologies, trade, and choose a way to overpower the other empire. This one is multiplayer.

The Rabbits mario RPG is Xcom MP I think? Maybe Google just in case though.

It sounds like you're actually looking for turn-based tactics/strategy? In that case maybe check out Civilization VI and the Advance Wars 1+2 remake. They're both supposed to have local multiplayer on the Switch, although I don't know how good it is. I don't think you can play the Advance Wars campaign co-op, but it sounds like you can at least play individual battles like that.

If you're really looking for real-time strategy, Pikmin 3 seems to have a good co-op story mode. Pikmin 4 technically has one too but the second player isn't able to do much, so it's probably not very fun.

Some folks have pointed out X-Com is a turn based game, and I agree with the suggestions that fit that.

On the off chance you actually want a real time game, I’m a huge fan of the Kingdom series. The co-op one is called Kingdom Two Crowns. It’s an interesting take on real time strategy as it’s a 2d side scrolling game. You are managing a growing colony in a hostile land, and you have to build up your economy enough to construct a boat and sail to a new island. You end up having to double back to earlier islands as you unlock new tech, sometimes finding your village chugging along nicely, sometimes finding quite a grim scene in need of cleaning up.

It’s a very chill game for the most part, with some higher stress moments that are paced well imo. I personally love the art style and music.