Personally, I'd go with the XL. Since I play game devices with big screens all the time (PC, PS VITA) I'd [almost] always pick the device with the bigger screen, even if it's not the most pocket friendly.
Plus, from all of the reviews I've been reading, the 3D effect on the XL doesn't dissapear as easily as it does on the original 3DS when you tilt the device and all that. The XL also has better curves (smoother corners) so it won't dig into your hands as much as the original. And it's also better if you have big hands like me.
But that's just my opinion. There's always gonna be someone out there who would think otherwise :)