Video games are no different from movies, books, TV, etc. They're just an entertainment medium - with no age limit. You don't hear people saying "You should stop watching TV, you're getting too old," so why should video games be different?
The only reason people think that way are because video games are still a relatively new entertainment medium. As time goes on and the people that have grown up with them get older, that mindset will begin to die off.