Stephen Wolfram explains the significance of a computational language in being able to represent the world computationally and how this is similar to the advent of mathematical notation. He highlights the goal of his work in building the Wolfram Language, which is to enable the expression of computational thinking in a way that can be understood by both humans and machines.