Nearest neighbor is a classical concept in machine learning that involves computing the closest point in a dataset to classify a new point by measuring the distance between them. This approach has evolved into more complex methodologies, but the core idea remains relevant.