The deer population can definitely get out of hand without hunting, but that might be because of the shortage of wolves. Hunting should always be regulated carefully by ecologists.
Fishing, I almost don't ever feel bad about fishing giving the nearly ubiquitous presence of well...fish. That is the only form of hunting I partake in personally, and fish don't share a similar social intelligence as deer. I will eat venison, but I don't have the heart to kill a deer. Call me a hypocrite.
I do wonder about the ethics of killing intelligent animals, especially pigs, which are debated to be smarter than dogs. Why is it ethical to kill pigs and not dogs? Though, again, fish, and fowl don't have those intellectual capabilities and therefore I don't even begin to think of killing them as being an issue of ethics. Now, why should intelligence matter??? I don't know, it's a matter of possessing qualities of humanness which probably makes it more difficult for me to hunt mammals, though that doesn't seem like any real justification other than having sympathy.
This is why I don't have a real stance on this issue, I'm not sure what to think as far as ethics go since I usually only think of human interactions.