Well Matt kinda just fills a side-role without actually getting too involved in the plot. Tyler was central in season 3 and the beginning of season 4 until he was written out. I really like him though, what with being the first successful Klaus minion hybrid heh.
But yeah they're just extra characters. Come to think of it, we haven't seen much of Bonnie lately either. I really liked her in season 4, she's this really great rouge witch now (:
I know right?! I always hated human Elena with her hypocritical attitude and what not. Right now she's a much deeper character, at least in my opinion. Not to mention that Klaus being as weak as he is right now in front of Silas is something I'm really enjoying.
You liking season 4 any better than 3? Because season 3 was really messy with dozens of sub-plots and what not. Personally, I think this singular-plot focus that's going on at the moment is really nice. Plus, it's getting really suspenseful ^_^