Don't be afraid if you don't have an audience to begin with. Many people with massive audiences have failed in podcasting, so start with low expectations and be open to learning and improving as you go.