The speaker recalls several instances where they narrowly avoided dangerous situations at sea, including encountering other vessels too closely and the potential threat of armed guards on other boats.