The downside of holding Bitcoin is the potential for high drawdowns, as well as uncertainty and volatility in the market. This risk should be taken into consideration when formulating a long-term investment strategy.