I don't mind the lengthy response one bit. I didn't personally grow up with spin-offs so we will have different perspectives. I grew up with gen 3 and fell out of the series until 9th grade, where I was given Omega Ruby as a Christmas gift. Which gave me an affinity for Gen 6.
Yes, the core Pokemon series runs on a formula. Yet I still find these games to be some of the most repayable ever. It doesn't get mentioned much, but there's no limit to the amount of team customization. I've always felt like playing with a different team feels like playing a different game. For example, I played Platinum for the 2nd time using an Ambipom. I never used it before, and it was sort of like a team MVP. The new experience of using this made the replay worth it for me. This is where I will fault Mystery Dungeon. I'm doing post game stuff, aware this game can't be replayed with new enjoyment. It follows a story and there aren't any branching paths. The most customization I would get is picking a new starter and partner. Since it's story based and there isn't enough changed dialogue, that isn't enough to warrant a new play-through. But if I wanted I could load up Emerald and start up a team of all water types and it would feel like another experience. I feel Sun/Moon hinders replay value, because if I want to go through with another team, it means going through massive amounts of dialogue where I know the outcome. I'm not saying it's a bad thing, but there is a sacrifice with the amount of dialogue, which is replay value.
I actually think that the community with Pokemon is unique, as there doesn't seem to be much of a desire for change in the formula. So take Gen 5 for example. It was different, but received some of the most heavy backlash upon release. I was around 10 years old during the release of Gen 5, and I remember the endless hate and criticism of "excluding" the old Pokemon from the main story. It was actually some of the criticism which caused me to fall out of the series. I was 10 so I took these opinions to heart, as why would I want to play something so many people hated? This same criticism came back with the release of gen 8, because some old Pokemon were cut completely, and people wanted the familiarity of playing through with old favorites. There was this brief period of time where (Serebii...I think?) released false information about the games including more than 8 gyms, which led to a happy community response. I saw complaints about the gyms being boring since they all use monotypes, but nothing about the "core" formula of going through the gyms to be boring, which I thought showed a lot about the community. With Gen 7, I saw most of the complaints about the characters being dull. Hau seems to be the favorite punching bag when these complaints arise. Then with Ultra Sun, I viewed most of the complaints saying that they didn't change enough from Sun and Moon, not that the core formula was too similar. Not my own opinions, just what I recall reading in various places from memory. Because no, Ultra Sun was not similar. The story takes a completely different turn halfway through so I'm often confused on how these complaints even exist. Unless they just picked it up, saw the early parts had similarities, and dropped the games. Yes the map was similar, but so were Crystal, Emerald, Platinum, etc. Yet these are considered the "best games" in the series. They change less than Ultra Sun/Moon did as those games I mentioned actually do follow an identical plot. Now in high demand are remakes of Diamond/Pearl, which would absolutely follow the gym formula. Which shows that it doesn't seem to be a big deal.
I went off topic a little, but when it comes to spin offs, I enjoyed Colosseum and XD: Gale of Darkness. I've finished Colosseum three times, and Gale of Darkness twice. Yet there's an issue with both of these games, and it's the Pokemon choices. Since you are confined to snagging, you are very limited in what you can pick for a team. Still, I disagree about the plots being good. You still have an evil team stealing Pokemon, just with the addition of turning them into shadow Pokemon. The motivation is an evil leader who wants control, and that's no different than the main series. If you tossed gyms in, made it so you catch XD001 after the 7th gym instead of the end, you now have the same formula. To me it felt like these games just swapped around the order of a few things and took out the gyms. Again there were other factors which hurt the plot. Wes blew up the team hideout, but you don't even get an answer why for the sake of keeping him as a silent protagonist. I don't find it to be well executed, because if they want to make a story focused game, they should go all in. Even in Explorers of Sky, they actually have the protagonist speak with the partner at the end. There's no reason Wes couldn't have explained his motivations somewhere along the line. There's that girl who follows you around who conveniently can see Shadow Pokemon. There's not any back story for her, just she ran into you and wants to save the Pokemon. The villains such as Nascour don't have back story. They just act and look evil for the sake of it. I don't even remember if they explained the motivation distributing of shadow Pokemon is money. I guess it can be implied that it's a black market of sorts, but there's just so much lack of characterization that the plot could have been much more. And this is what I've seen said for some of the main series games. Don't even get me started on the Gale of Darkness protagonist. You could throw him into a main series game since he's completely blank :/ He's a kid who has a sister and works at the lab. It's not any different from the main series.
I wasn't old enough to play Stadium as they came before I was born, and never saw a virtual console release. A relative with an N64 allowed me to see some of the features, I tried the Gym Castle and it felt like it was meant to get the old GBA Pokemon into 3D. I haven't played Battle Revolution either, but that's more due to lack of knowledge which caused no interest in buying.
I have played Pokemon Dash, and it makes me feel the same as Pokemon Snap. Since from what I've read you feel the sake of trying something different makes it more enjoyable, but it falls into a different perspective for me where I feel like if I played a main series game I would still gain more, as I find those games haven't become stale due to being able to customize a team. Of course it all falls to the same place, just different opinions.
Since I have to end this short (dying laptop battery -_- also I'm hungry ) I'll say I gained a different perspective from reading what you wrote. The spin-offs seem to be a playing field for the formula of the main series, since the main series doesn't seem to change much. Reason being the community might not respond well. I recall reading Gen 5 were some of the worst selling games at the time, which might explain some of the hesitation now to try something different. And if you get more enjoyment out of them being different, it again all comes down to opinion. But keep in mind the original word was "better" and I still don't think trying something different necessarily makes them better than the main series. As you explained in your own post there were certain features that were ahead of the main series, but at the cost of something else. Such as the customization but loss of plot in battle revolution.
