There's a spring loaded wristsheath for pathfinder that came out with the Adventurer's Armory-swift action to draw, costs 5gp. The quickdraw feat makes drawing a hidden weapon as a move action, you could tie the two together and make quickdraw+wristsheath=free action, but I think that gets into the territory of "do you really want your players to have that too?"
And if you're sliding the blades into outer pockets (as per the jacket suggestion-with the hilts exposed-they're not hidden weapons. You could just have a jacket and a vest, and a pair of bandoleers.
If you're talking standard throwing knives (which tend to be lighter and more slender), you could have four per side in the vest, six per bandoleer, (bringing you to twenty), and four per side in the jacket, with two on hips/wrists/whathaveyou.
Also-placing them along the arms, diagonal, hilts exposed for easy grasping.
Honestly, the only thing that keeps you from rigging a peice of clothing (hopefully of durable materials) to hold multiple throwing knives is access to a leatherworker/tailor, a suitable peice of clothing, and space limitations.
Obviously if you're talking about throwing around bowie knives/hunting knives you're going to run into space issues, but for game mechanics purposes there's no difference between a rondel dagger, a stiletto, a throwing knife and a hunting knife. They're pretty much all classified as daggers (for combat purposes)*. Reflavor as you like-but throwing knives range between four and six inches and are often rather narrow (about an inch across as a blade), with very small, minimal hilts. They're remarkably easy to carry in lots, especially if you don't have to worry about them rubbing you raw (worn over armor or leather-even canvas or boiled wool probably isn't going to cut it).
*yes you could classify the hunting knife as a "survival tool", and you could nitpick and put a limiter on size in order to do the appropriate damage-but that's GM fiat and just being difficult
Edit: and my apologies about the hidden thing, I'm used to the idea that the guy walking around bristling with knives openly is going to have some social interaction issues....but hey, it's legal to carry a sword around a modern city as long as it's visible*...so...
*did this one hallowe'en, police though it was funny I called to ask in the first place.