The speaker recounts their dissatisfaction with Warren Buffet's suggestion to invest in the S&P 500 or a low-cost index fund and express their preference for lower risk investments.