Well, the readings seem to be:
-Most Favorable Order(caster): Ill Omen -> 2 rolls -> Misfortune -> 4 rolls
-Most Favorable Order(victim): Misfortune -> 2 rolls -> Ill Omen applies to one -> 3 rolls
Most Favorable Order has clear problems, beginning with who gets to decide best?
-Last In First Out: Last effect applied chronologically goes first, then stack from there.
-First Come First Serve: Effects generate rerolls from the first, and to all subsequent dice the same applies.
-Apply both to Base: Ill Omen +1 roll pick worst, Misfortune +1 roll pick worst.
In this scenario, each 'reroll state' occurs in an atomic instance(i.e. other than the reroll effect itself, nothing can see the additional dice).
So for example, you rolled a 16, which causes one of them to trigger(say Ill Omen), rolling a 13. Then Misfortune triggers, and sees the dice rolling 13, and performs its own reroll, obtaining a 10 and replacing the dice again.
This does have issues with positive rerolls(since order matters when the rolls are NOT equivalent) and worse, 'pick one' rerolls.
Personally I'd go with each effect only seeing one dice, which is the combined state of all previous dice. Then have positive and negative rerolls cancel out, and choice of roll effects go first. This generates the fewest additional dice, and makes things go more smoothly, mitigating the urge to smack the player on the head for this.
The rulebook itself doesn't give you precedence rules anyway.