The speaker explains their difficulty in remembering things that are not beneficial to them and the reliance on other people's memories to fill in the gaps.