The speaker advises against linking confidence to ability, instead suggesting that confidence should be linked to one's intentions. This allows individuals with beautiful intentions to have the confidence they deserve.