The speaker recounts their fear of losing $100 and how it felt like a lot of money at the time because they had nothing. They were also scared of the card that told them to leave town.