Only Unarmored Defense, Martial Arts, and Unarmored Movement forbid wearing armor. If you're willing to forego those features (and plenty of multiclass builds already are willing), then nothing is stopping you from doing it.
I wonder how it looks when the Kensei uses a bow and them pummels the target he shot for bludgeoning damage. Does the back of the arrow snap off and spin into the target for more damage?
e: Let's actually do some math.
Level 17 Kensei
Greatsword with GWM = 2d6+10+Dex twice = 34 + 2xDex average
-add pummel = d4 = 2.5 average
-add Flurry = 1+Str twice = 2 + 2xStr average
Shortsword = 1d10+Dex twice = 11 average + 2xDex
-add Flurry = 1d10+Dex twice = 11 average + 2xDex
So flurry vs. flurry, we have: 2x(2d6+10+Dex+1+Str) > 4x(1d10+Dex)
2x(7+10+Dex+1+Str) > 4x(5.5+Dex)
36+2xDex+2xStr > 22+4xDex
14+2xStr > 2xDex
7+Str > Dex
If you have GWM, you have at least a +1 Strength modifier, so the above holds true until you get +8 Dex, or 26. However, this isn't accounting for the higher accuracy on the shortsword routine. Plugging in +1 Str and +5 Dex, the Kensei routine is only 3 damage higher at the cost of accuracy.
If you want to eliminate the Strength parts of this because the accuracy would be comparatively bad:
GWM + Pummel vs. Flurry
2x(2d6+10+Dex)+d4 > 4x(d10+Dex)
2x(7+10+Dex)+2.5 > 4x(5.5+Dex)
34+2xDex+2.5 > 22+4xDex
14.5 > 2x Dex
So that comparison still holds true until +8 Dex. Again at +5 Dex, the damage differential is only 4.5 in favor of the Kensei for the cost of lower accuracy, though this one doesn't spend a ki point vs. the shortsword needing one to fall just short. Using Martial Arts for the shortsword's bonus action instead of Flurry of Blows widens the gap to 15 in favor of the Kensei. The Kensei's advantage does not seem to be damage, but rather its damage not falling off as badly when you're not using ki points. The shortsword's advantage is higher accuracy for a slight trade off in damage and effectively having the option to wear a shield.
So after that mathematical analysis, I'm not really impressed by the Kensei as a stand-alone. It's a low-level hero that tapers off and doesn't have nearly the burst potential of an Open Hand, Long Death, or Tranquility monk at high-levels. The gimmick really boils down to accuracy control, and without GWM, it doesn't really hold up.
Level 3, no GWM, flurry vs. flurry:
2d6+Dex+2x(1+Str) > d6+Dex+2x(d4+Dex)
7+Dex+2+2xStr > 8.5+3xDex
0.5+2xStr > 2xDex
0.25+Str > Dex
So without GWM, as long as the Dexterity modifier is *1* higher than the Strength modifier, a Kensei weapon is less damage than a shortsword. A quarterstaff scenario forces the Strength modifier to be higher, at which point, why are you playing a single-classed monk?
Level 3, no GWM, Pummel vs. Martial Arts:
2d6+Dex+d4 > 2x(2d6+Dex)
7+Dex+2.5 > 7+2xDex
2.5 > Dex
So at level 3, the Kensei has an advantage if the character doesn't already have 16 Dexterity. A quarterstaff reduces that to 14.
At level 5, no GWM, flurry vs. flurry:
2x(2d6+Dex+1+Str) > 4x(d6+Dex)
2x(7+Dex+1+Str) > 4x(3.5+Dex)
16+2xDex+2xStr> 14+4xDex
2+2xStr > 2xDex
1+Str > Dex
Level 5, no GWM, Pummel vs. Martial Arts:
2x(2d6+Dex)+d4 > 3x(d6+Dex)
2x(7+Dex)+2.5 > 3x(3.5+Dex)
14+2xDex+2.5 > 10.5+3xDex
5 > Dex
It looks safe to conclude that without GWM, the any other melee Kensei weapon isn't better than shortsword for single target damage past level 8 when a character could reasonably expect to have 20 Dexterity. Using a quarterstaff, the character only needs 18 Dexterity to do better on average, which is reasonable at level 5. The non-shortsword options for Kensei weapons won't keep up without GWM.