The speaker suggests letting go of man-made goals and figures by focusing on doing the best you can, rather than aiming to measure up to a made-up number. By stripping back and recognizing what really matters, intrinsic motivation and appreciation of the craft can be enough to push you forward without being tied down to unrealistic expectations.