There are some bardic prestige classes without casting, like war chanter and seeker of the song. You could play a bard build with just one actual bard level and not much casting in the first place. If you want actual bard levels, and your DM allows, trading casting for a feat every two levels seems reasonable IMO.