I'd warn that it would make your intimidate checks less effective against creatures with their own scores in Intimidate, which potentially could be "all of them", but I ultimately have no problem with it.
Just a word of warning so that you know you're potentially asking to nerf yourself so that you can resist Intimidate better.