The speaker realizes that thinking about money makes him anxious and he prefers investing in things that make him happy but don't require a lot of money, which led him to make all his investments passive.