The speaker suggests long-term investors to have patience, stay calm, and add to their position during the dips. He advises people to look at the S&P 500 of the past 10 years, which has endured several crises and has still performed well.