For the PC, it was a Reversi game that was on a 5 1/4" floppy which came with the IBM that ended up being my first computer. I was two years old, and I'm pretty sure 2-year-old me could beat 21-year-old me at Reversi--that, or the AI in that game was just really bad. (Actually, that one's more likely.)
For handhelds, it was Super Mario Bros. Deluxe for the GBC, when I was 10. I got amazingly lucky and found the game and a GBC at McDonalds, and we stayed there for over an hour and no one came to claim it, so I got to keep it.
For consoles, I have no idea because I used to hang out in electronic stores just to play on their displays. But the first two I owned (got them at the same time) were Creatures for the PS1 and Rayman Arena for the PS2, when I was 12.
...It amazes me that I can remember tiny details like this, especially from so long ago, yet I can't remember what I had for dinner two nights ago.