Me too. I had bright blue hair once, the most crap i took was from the guys at work, everywhere else I just managed to silence people with its brightness
But at work they called me a parrot, bastards.
So I agree. I think you should wear what you want, if you don't want to stir attention then don't wear something that will, it's that simple.