Hopefully I'll be able to give a more detailed answer later today or tommorrow.
Really, the tl;dr is everything form weapon damage to cost should have had a balancing formula to it. Like cost specific, if you did say 1pt ammo, 2pts sensors, 4pts basic equipment, 6pts refined, you could balance all your costs. Later in post, you can inflated it to a 50~300 point system since the numbers them selves are arbitrary. But if you decided something should cost 225pts it should equal to something like a [basic (4 or 200pts) + 1/2 ammo (0.5 or 25pts)] and if it's not then you know it's cost needs a different value.
Eeerr, Arcane Pilot has plenty of Divinations, from the humble Detect Magic to Locate Object to Clairaudience/Clairvoyance to Locate Creature to True Seeing, several more all the way to Foresight.
The core problem is you started at the SRD list and then started cutting Spells out which ended up leaving a theme of patchwork (no scry until lv14?). Like you have Alarm sure, and all the other upgrades for personally sleeping safely at night but you can't Fabricate a series of walls to protect more than a half dozen people or Move Earth some trenches in for warfare. What you have is a mech sized magical plate that can hold dirt the caster's mech sized bare hands have to pick up using his mech sized shovel that deals minimal damage to dirt.
Some of the cuts make sense for balance sake, but if you're looking to balance the List then it also needs to be balanced to what the other Classes can do. Like yeah maybe the Arcane Pilot gets the same bonus Feats Super/Real does, but so does the Real/Super and that's not going to change the fact the Real/Super have exclusive extras for attack countering. The Arcane (and divine) should also get their own alternatives too, specially if you feel players
need it to begin with.
Why yes, the intention is for Arcane Pilots to specialize in a certain type of magic instead of every mage character knowing all of the uber spell ever without real effort. And any of the spells you mentioned are enough on their own to make the Arcane Pilot a MVP for the party.
>.>
You have a funny way of wanting Arcane Pilots to specialize in a limited report of Spells. For one thing they can't choose a Spell that supports their specialty until a level after they gain the new Spell Level and "Favorite Spell" always shows up a level before you can choose your actual favorite Spell so you always "favor" spells you hate using.
Then as you're doing in the game you can get extra spells through a variety of items.
Yeah but Phantasm Star is a high powered game, some List snatching is expected, and Super Robot Wars the Campaign setting is not forced Gestated & high powered.
That being said you can set aside the differences in expected scale and look at the heart of it. Because I had to invest I wanted to increase the return, one thing led to another and in cause and effect Bahamut really doesn't have anything you'd call a cool item, since he's just a dragon on fire that spent most of his wealth trying to obtain Spells Arcane Pilot doesn't have.
And that's ok in the build. The RPs been fun, next level I'm a lot less constrained with my buffs with got me to pick some more interesting ones which are still pretty new to me. But Bahamut & Phantasy Star aside, Arcane Pilot has some problems. Even the Warmage, Beguiler, and Dread Necromancer gets Splat Spells. If you want the Arcane Pilot to really narrow in on some favorites, less choice isn't the solution but more. Like you could cut the default list some more so it's purely basic role fulfilling then triple the open space giving them at least three Spells of every level to pick.
When people are limited they always grab what can give them the most. Like in another lovely SorO analogy. If you're poor you walk into Walk-Mart and throw a bunch of Hamburger in the cart because you can cook that in all kinds of ways and Hamburger Helper literately sells pasta ideas you can create after visiting the baking & spice section. But if you have the money to spend, you might instead walk in and think I want Tacos and BBQ New York strips which just doesn't have Hamburger in every option the guy with more restrictive limits has. You don't
have to pick Polymorph's ominous solutions if you can pick up a number of unique Transmutations you like. You don't
have to pick up Orb of Acid if you have the spare room to pick up different elemental Spells or even more than one area one Spell. Etc.
You'll always have a narrow section of people that will try to abuse the system much like the rich guy that refuses to pay his taxes, but choice breeds the ability to select things that do not amount to the best swiss army knife.