First off,
As DM you set the campaign world thats true, but you want to make a world that is enjoyable to your players. Giving some leniency about characters and what they create is good, but the best way to deal with players that want to play evil is to talk to them like they are adults and let them know that evil will face serious disadvantages in the coming campaign(just let them know they will spend most of thier formative levels in jail or get hanged and they'll get the picture). Most people don't want to run an evil character in a good campaign.
Second, and perhaps on an opposite note, I think you should run an evil campaign. I ran my group through an evil campaign with the setup that we had been playing in a particular created campaign world for years and everyone was ready for a change. It was a collaborative world that each of the people who DM'ed had fleshed out a particular area etc. Eventually, the 3 DM's in the group felt that we just didn't have anywhere else to go with this world and decided to get rid of it. So we had our players create evil characters with a "4 horsemen of the apocalypse" theme and actually had some good roleplay.
I should mention that there was a significant amount of intervention by powerful evil deities and the group only went from 1-10th level before we destroyed the world and moved on to a new campaign. I feel we spent a good amount of time exploring the dark side though, and I actually found DMing an evil campaign around the same difficulty as DMing a neutral party. A DM doesn't need any lofty motivations for evil characters, simply something that offers more power, or something evil and more powerful than them (of a kind that character likes) is usually enough.
The point is, I gained some significant experience as a DM in knowing how to handle evil characters, and it helped me work with one-off evil concepts in future games.
So now I don't balk if a singular character wants to be LE or even CE in a good campaign, there are ways to make it work. Of course the party majority has to be in line with a particular concept, evil or good, depending on the campaign, so one-offs yes, but majority evil with a good setup doesn't work. Thats why you have to let players know that the alignments they are picking will not work..
PS. Allowing my group to play evil, made them miss good. The next game we ran there were no neutral or evil characters, people will come to miss being good, despite themselves