The speaker emphasizes the danger of making wrong inferences and predictions about others' feelings and thoughts due to experiential blindness, which can cost lives in the medical field. Different cultures have varying practices in making predictions about someone's actions without inferring anything about their internal state.