Unless you don't have an option to go to speaker phone, or just really love striking a conversation with random people, waiting in line will always be worse. Once I'm told my call will be so long, or even before then, I just hit the speaker button and go back to whatever it was that I was doing. And if you need to move to a different area, take the phone with you if it's cordless.
The only thing that makes being on hold aggravating is if you get connected to someone who doesn't help, hangs up on you, and forces you through the process again. That's painful. At least in person you can ask for a manager or a different representative. On the other hand, you aren't subjected to a bunch of roll eyes and people huffin' and a puffin' behind you because you're holding up the line to get your money or your time's worth (really stupid questions are another thing entirely, though).
I guess the answer depends on how much of a social person you are (if you'd rather avoid the looks that may kill, being in line for service issues is never going to be fun), what the line is even for (amusement or service), how restless everyone involved is, and if you have other things to do while on hold over the phone (though honestly, you can find stuff to do).