The speaker recounts how they struggled with feedback because they felt a strong sense of responsibility towards anonymous individuals who provided it, and because they haven't received much positive feedback before. They felt paralyzed and unable to fix specific complaints or act on them.