The speaker realizes that despite being raised in a non-religious family in New Hampshire, their Jewish identity was still very important to them, as it was the identifying factor that set them apart from their community.