The speaker recalls their early fascination with making money through ventures such as selling seeds from a local tree, and their childhood belief that one million dollars meant having endless money.