Normally around Thanksgiving is when I "traditionally" start to get more into Christmas stuff and whatnot (since then or a few days after is when my family would normally put up the tree, depending on how lazy we were feeling). I don't really get annoyed if other people decide to start feeling it earlier, though if it's too long before the end of November I might be all like "why are they doing it now??". :P
I won't really get on their case about it, though. If they want to start getting into the spirit of it before I do, they can.