I'm late to the party on this conversation, but i wanted to chime in. I do either thanksgiving night or Black Friday. It's a neighbor respect thing for me. What I could have done the first year is different now that I'm in year 5. Don't get me wrong, the neighbors love the display - TONS of people commented / stopped by to say hello during setup.
You likely have more leeway before the holidays than after as well as the novelty factor. I would encourage you to keep all neighbors happy (within reason). It's really nice to have the respect and support of neighbors if you're in this for the long run. If they turn against you, it's really, really tough to get them back on your team. (We made cookies for our 8 closest neighbors last year as a good-will gesture.)
You are correct in stating its your home / display, however, there are several instances where neighbors got fed up and took the homeowner to court. That costs big bucks. The shocking thing- the neighbors won. The display was a "public nuisance".
Obviously your choices to make, and certainly do what makes you smile. I would urge caution if you are hearing comments from neighbors. Remember, you only get to hear comments from those bold enough to say something in front of you. What do they say when you aren't a part of the conversation?