Ultimately I think people dislike optimizers for the simple reason that many optimizers are not considerate of what the rest of the party wants, and don't know when to reel it in.
If your party's Monk is doing 2d6+8 damage an attack, your Ranger is doing 4d10 with Manyshot, and your low-op Wizard's thing is tossing out 10d6 Fireballs, and you bust out your ubercharger build that does 2K damage per round, I think you're being selfish and kind of a dick. Because of your build, the other players contributions are no longer meaningful, and when that's the case they're not having any fun. It also becomes a nightmare for the GM, who has to balance for 3 non-optimized players and one super-optimized player.
Occasionally pulling out a trick you discovered that one-shots the encounter can be fun and entertaining for everyone. Using that same trick repeatedly rapidly loses its charm.
And if your group having fun means that you need to rein yourself in a bit, so be it. Save that ridiculous charge for when your party needs it, and otherwise go with less-optimal options. Maybe come up with a back story for why you don't just OP every fight away. You'll still have fun. Seriously.