The speaker is not concerned about getting banned from TikTok due to the toxic nature of the content, but worries about the impact on his podcast. He also suggests an easy way to regain followers after being banned.