Well, I'm presuming a justifiable magic stigma, depending on the setting.
If high level magic is rare, then a mob can still take down a level 4 caster, and by doing so maintain low magic.
Magic as useful and as powerful as it is, the effect of a lynch mob level stigma would only be to reduce the magi to those who master battlemagic, the ones with only non-combat spells would be history.
So, magic being taboo would require cultural, political/religious or practical reasons.
Culture may involve history(former mageocracy, fell and then the people decided not to ever let magi rise to power again, on the basis of killing them while they're still weak) or identity(stereotypical warrior race pride, where magic is unfair and unclean, especially if foreign in origin). This depends on cultural values to enforce, and generally would be more indirect.
Politics and religion is simpler. A force in power would very much like to be the only ones who have power. Outlaw magic amongst the lower classes, or outlaw certain forms of magic. This has actual enforcement power, since doubtless the enforcement is performed by trained armed forces and has its own sanctioned magical backup.
Practical needs to be tied to the world's actual metaphysics. Does being a spellcaster make you statistically more likely to be evil, dangerous and/or insane? This is self policing, the natural law would screw the mage up more than the people ever will.
However, theres varying levels of stigma(the lynch mob being on the extreme end).
What if, instead of being assaulted, the known mage is shunned by default. People would go out of their way to ignore or bar his way, but stop short of doing violence barring a panic(like say, a child goes missing, everyone blames the wizard, and after a few rounds of beer you have a lynch mob, who's only crime is to be drunk and afraid).
This I think, would be better off than the full out kill-on-sight method, which really takes an organized force more than a mob, and require that spellcasters be sufficiently rare that the loss rate is acceptable for the society.