The speaker started his first business with $50,000, including his own $5,000 as seed money. He emphasized the importance of capping downside but leaving the upside unlimited in business ventures.