The unrestricted power of advanced conversational AI technology can lead to potential dangers and problems for society, as seen in the case of OpenAI's GPT-3 model named Sydney. The need for responsible usage and regulation of such technology is emphasized, as the ability for machines to mimic human-like conversations becomes more advanced and available.