I understand that a lot of people are concerned about the quality of these games after Game Freak's statement(s) and are pretty much perfectly willing to shred every trailer from here on out for evidence of "high quality animations", but outside of social media where a typical parent is debating on a birthday present for their kid, this kind of controversy would barely reach them (if ever), and will be the bulk of Pokemon's purchases.
So no, SwSh won't tank. Unless you know millions of other people that also won't buy the games, it's safe to say these games will be fine and will sell just like any other.
As far as what happens afterwards? Obviously no one knows, but one can be hopeful that Game Freak takes into consideration fan "feedback" (for lack of a better term) moving forward as far as working on their next Pokemon project is concerned. Maybe they're waiting to see how Town does in sales before jumping full-force into Pokemon gaming again.
If they fail, then it's doubtful that Pokemon main franchise would end, because regardless of how many copies sold, it's still a huge money maker for the franchise as a whole. What people need to understand is that the games are just one facet of Pokemon; there's the merchandise, the anime, the manga, the TCG, etc. The games, even if undersold, can as a promotional figure for all the other facets of the franchise. 5 million SwSh copies sold might be a "failure" in the grand scheme of things, but that's 5 million perhaps extra people that might be encouraged to buy more SwSh-themed merch to make up for that loss, so there's that.