The speaker reflects on his skepticism about certain stories presented in memoirs, suggesting that certain elements should be categorized as fiction from the beginning.