Hey, you know... Lesser Restoration also removes fatigue, which is what lack of sleep causes. So for those non-spellcasters who are planning to chug a potion anyway, they can not sleep and quaff the potion in the morning to get rid of their sleepiness.
So I think Nordar will be doing that, regardless of whomever is sleeping/not (unless we have three other people with more than 3 points of Con damage...)