It's not your imagination, actually! All Pokemon have a rate at which they gain experience. You can see it on Serebii, actually, it's right under where the abilities are listed on the Pokedex page. It says Altaria has "erratic" growth, and it only need 600,000 EXP points to reach level 100, while Blaziken needs way more and is classified as "medium slow." Not sure what other mons you are using but chances are that Altaria is faster as well!