The speaker prefers taste over judgment because judgment implies there's one answer. He shares an example of how identifying certain outliers can be helpful in hiring.