The speaker shares their experience of not knowing whether they were successful or not in the beginning stages of podcasting, emphasizing that success often comes down to being the right fit for the audience.