As I indicated in an earlier post, I think this escalation-style agreement isn't bound to work. The idea is to just remove these things from the table. The mechanism used to do so was some variation of Mutually-Assured Destruction. But, there are innocent bystanders (the other PCs) involved.
Ideally, the response would be to say something to the effect of "you know what, that really should have been banned, that was kind of the whole intention of that list, was that they'd be banned except under special circumstances. But, I didn't want to come out and ban a whole bunch of things b/c I feel bad about doing it as a DM. Sorry." The player's response, taken out of context a few pages ago, did sound petulant. But, maybe if things are made clearer -- that it's essentially an arm's race and that's not likely to be fun for everyone -- then maybe his response will be different. Also, he may not recognize how outpowered or problematic Shivering Touch is.
Personally, I'd advocate against kicking the players in the teeth to prove the point. That'd be my last resort option, as it's bound to create more antagonism rather than defuse it, which I take it to be the goal.
Finally, it depends just how much is riding on all this. If he's going to memorize 1 Shivering Touch a day, that's probably something that won't swing too many encounters. So, you could take this as a "teachable moment" for the group and revisit the approach to your particular gentleman's agreement and perhaps leave him with Shivering Touch anyway.