To-hit - Dex instead of Str

[ImperatorK]

What effect would this houserule have on the game, other than making Dex too loaded?
Also, what could potentially be added to Str as a replacement?

[ketaro]

Every str modifier divisible by 10 effectively makes your weapon do damage as if it was 1 size category larger?

10-29 (+0-+9) medium weapon does normal damage die.
30-49 (+10-+19) medium weapon damage die increases as if large weapon.
And so on.

If your players aren't the type to minmax a 40 to 60 in an ability score by level 10, it'd be totally viable to downscale  that to every +5 modifier above 0.
Atleast to me, that makes str seem really worth an investment as it really is not applicable to much anything useful anymore one you take away it's to hit synergy.

[ImperatorK]

That's an interesting idea. I remember reading somewhere about a houserule for martials that improved their weapon damage based on BaB. Maybe if I combined your suggestion with that? But first I'd need to find that houserule, 'cause I don't remember the specifics...
It would be something like - you multiply your weapon's damage by your BaB, but only up to your Str mod.
So if you use a shortspear (1d6), have +4 BaB and Str of at least 18, you deal +4d6 damage, but if your Str is 14 you only deal +2d6 damage.

[RobbyPants]

What effect would this houserule have on the game, other than making Dex too loaded?

At first glance, this is a straight-up boost to anyone who wouldn't be using Str, anyway (wizards, sorcerers, bards, and rogues), detrimental to "standard" cleric builds (but they could be built around this), have minimal effect on druids, and straight-up kick most martial characters in the nuts.

So, my first question would be "why do you think you need to boost wizards and kick martial characters in the nuts?".

A more detailed look:
The too loaded thing is the biggest thing. It's already a pretty good stat, but this change could basically make it so it could never be dumped, except by completely non-martial characters.

Fighter-types are already three-stat MAD, before you start factoring in mental stats for classes like Paladins, Sword Sages, and Hexblades. Often times, people will dump Dex if they can get away with wearing full plate. This strategy will no longer be viable. In fact, heavy armor becomes far less useful when you figure how the max Dex of an armor interacts with Dex-to-hit. You'd likely have a higher AC with light or medium armor than you would with plate.

Power Attack builds might become harder to pull off, as the stat that affects attack success is now separate from the damage stat. Power Attack already is pretty fiddly with optimizing your ability to hit when sacrificing much/all of your BAB. Now, this rule would allow you to have a slightly higher AC when using Shock Trooper, but it's likely not going to be a big enough difference to matter.

One pro is that AoO builds utilizing Combat Reflexes will see a boost.

Also, what could potentially be added to Str as a replacement?

It depends on what your goals are. If you're looking to make Str hard to dump, you could just nix Con and roll it into Str. This leaves you with a weird number of ability scores, but that doesn't matter for much of anything, strictly speaking.

Every str modifier divisible by 10 effectively makes your weapon do damage as if it was 1 size category larger?

This is a pretty complicated way to effectively say that "Str contributes even more to weapon damage". An easier way to do that would be to multiply the bonus damage from Str by some set amount.

In either approach, be ware of what it does to monster damage rolls.

[ImperatorK]

I like to inject realism into my games, but only if it improves the game and mechanics and is overall beneficial to the PCs (example - throwing improvements based on Str).
With that in mind, I guess I'll scrap this idea.

[RedWarlock]

I'm doing it in my rules, but they're a really extensive ruleset. A lot gets shifted around to even out the usage.

Str gets Fort saves (which become more like PF's CMD as a physical stability/grapple save), and adds to move speed. Dex gets deloaded (init and most ranged attacks go to wis). EVERY stat has effects on character choices, not just casters or martials or skillmonkeys. Plus, my ruleset doesn't have stat-replacements so much. AC uses dex and wis together, and DR is also important, given that there are plenty of automatic damage/damage-on-a-miss effects.