Except for my time on the Pokemon MMORPG, I've never ditched my starter in any time during the story, and it has always stayed on my party. Even if there are Pokemon that would be much better for my team, it's just that special connection you get to a starter from using it all the way through, though of course that wouldn't be there if it was boxed as soon as you could catch a new Pokemon. Either way, starters always have a reserved spot in my party, and I can't recall anytime except the one I already mentioned where I'd ditch my starter.