The speaker reflects on his childhood as a "two-day-a-year" Jew, only attending synagogue for Christmas and Easter and focusing on other interests like astronomy rather than his Bar Mitzvah.