I'm a GT fanboy and I really, really love some aspects of GT5.
But in a lot of regards, GT5 is a disappointment. Lots of cool features that aren't fully implemented or utilized... day/night cycles! Only on certain tracks. Dynamic weather! Only on certain tracks. Cockpit view! Only with certain cars. Track creator! That's actually just a semi-random track generator. The list goes on and on...
So overall, my vote has to go to Forza 4. The graphics still aren't quite as pretty as the GT series, but they do have some advantages: Solid 60 FPS, and none of the absurd levels of screen tearing that you see in GT5. The car selection isn't quite as big in raw numbers, but overall the variety feels greater due to the fewer number of duplicates. The physics in Forza 4 are improved over previous iterations. I personally still prefer the feel of GT's physics, but Forza's may be more realistic.
I sincerly hope that Polyphony Digital one ups Turn 10 with Gran Turismo 6... If they fix all the things that are wrong with GT5, it will be a truly astounding experience.