I'd say Gyms are formed as a requirement of the Pokemon League, they establish 8 and use them as a training and qualification system for the regions trainers, as well as for the Leader to keep a responsible eye over his community (sort of like a town mayor, I guess) Imagine it like this, The League are admins, Gym Leaders are moderators :) Gym leaders are probably picked from societies trainers for being particularly skilled, knowledgeable and having good leadership skills and a positive approach to training Pokemon. Experience with the local area would be preferred, but as Sydian said, not essential, i.e Norman. Flannery's Grandfather could probably influence the League, but i'm sure she still had to earn the position.
Why do some towns have Gyms and others don't? Small towns are usually not on a major transit route, so they can't support one. Although Lilycove and Slateport Cities in Hoenn are huge, they don't have Gyms, perhaps they don't need or want one. Gyms are so important in the games but their origins are still pretty mysterious and unexplained...