The speaker expresses their dislike for roasting, but suggests it would be more appropriate to roast someone famous instead of directing insults at others. They remind their audience not to "punch down."