The speaker encourages pursuing a passion because even if it's for a non-traditional field, the experience gained from it is valuable. Some may not end up pursuing it as a career, but it is still worth considering for the experience alone.