Yeah, primary spell caster or shapechanger is the way to go. Honestly, you should be fine if you wanted to play the High Elven Conjurer. Your save DCs won't be great, but ... who cares? I don't even think you'd need Polymorph, just page through the God Wizard handbook.
Dragonfire Adept isn't bad, either. And a sneaky Wizard is another high-survival build, though a little more complicated (especially if starting at 1st level). Really, all the suggestions in this thread are solid.