As a non-smoker, I find it disrespectful for a smoker to light up in front of those who do not want to breathe in their smoke in an enclosed or semi-enclosed space, and worst in front of children who do not know any better (and sound like a frog by time they are five). That is infringment of a right to good health. It stinks, it makes some people feel sick and get headaches, and even if you yourself don't smoke the smell clings to your clothes.
As a non-smoker, I find it disrespectful for another non-smoker to dictate that other people should not smoke and that smokers are low, bad people. That is bigotry and prejudice.
As a non-smoker, I do find it irritating that every 3rd person average I pass in the street is holding a smelly cig and that I have to hold my breath so many times in order to not have the smell sting my throat and nostrils. But I feel powerless to escape it.
My views on smoking, I think, would be different if non-smokers did not have to inhale the smoker's leftovers. I am fine with people doing it if they wish, and I realise that outdoors it cannot be helped smelling it everywhere I go (though I worry what that, coupled with car fumes, is doing to my lungs), but I do think its something that's not necessary nor healthy.