I'd translate them into this system, rather than dealing with them - if you're going to do something, do it whole hog.
You could go with 1d4-1d6-1d8-1d10, then reset to d6 and add 1d6, if you don't like d4s. That gives you a similarly smooth progression (I didn't go with it because using the d4 once and never again doesn't feel aesthetically pleasing).
For size categories, I just whipped up the following:
[Size Diff] is how many size categories larger/smaller than you your opponent is.
• You deal [Size Diff]d6 bonus weapon damage vs. smaller opponents, but suffer a -2*[Size Diff-1] penalty to attack rolls against them. You get a +2*[Size Diff] bonus on all Strength-based checks made against smaller creatures.
• You get a +2*[Size Diff] bonus to your attack rolls against larger opponents, but lose [Size Diff-1] dice of weapon damage against them. d6s are always lost first. You get a +2*[Size Diff] bonus on all Dexterity-based checks made against larger creatures.
• Treat objects as "creatures" of a similar size - rather than gaining bonuses to certain checks, however, they apply that bonus as a penalty to such rolls made against them.
So...
An Ogre attacks a human. The Ogre deals +1d6 damage to the human, but the human gets a +2 bonus on attack rolls against the Ogre.
That same Ogre wants to smash a door sized for humans - it gets a +2 bonus to the roll. If it tried to pick the lock, it would suffer a -2 penalty instead.
Maybe something like...
Fine: 2½'x2½'x1¼' space, 0ft reach, x1/16 weight and carrying capacity
Diminutive: 2½'x2½'x2½' space, 0ft reach, x1/8 weight and carrying capacity
Tiny: 5'x2½'x2½' space, 0ft reach, x1/4 weight and carrying capacity
Small: 5'x5'x2½' space, 5ft reach, x1/2 weight and carrying capacity
Medium: 5'x5'x5' space, 5ft reach, x1 weight and carrying capacity
Large: 10'x5'x5' space, 5ft reach, x2 weight and carrying capacity
Huge: 10'x10'x5' space, 10ft reach, x4 weight and carrying capacity
Gargantuan: 10'x10'x10' space, 10ft reach, x8 weight and carrying capacity
Colossal: 20'x10'x10' space, 10ft reach, x16 weight and carrying capacity
Numbers and such could stand for tweaking - I like the neat symmetry of "+1d6 damage and +2 Strength checks per size category, and a -2 to hit per category past the first" for large creatures, and "+2 to hit and Dexterity checks per size category, and lose a die of damage per category past the first" for small creatures.
And a general rule of thumb for the table above: each size increase doubles your weight, carrying capacity, and the shortest side of your space. Your reach is just the "middle" dimension of your space, rounded down.