Up until recently, it was my sacred rule to never ditch the starter until the end of the game, and to steer clear from challenges that required me doing so. Some of my latest playthroughs have made me reconsider that rule, however; I kind of wish I'd ditched my Totodile in my last Crystal playthrough because I actually couldn't bond with him, but the little bastard was too useful to let go after he evolved and learned Ice Fang. I also found it infuriating that having the Starter in the party was mandatory to progress the story in Xenoverse; to this day, it remains one of my greatest pet peeves in a fan game.
If you ask me now, I'd definitely dump the starter in any of the Sinnoh games in favor of an older-generation Pokémon. (PLA wouldn't count though, as the starters are already from other regions and I could never part with any of them, especially Cyndaquil!)