Pat grew up with pride in his Confederate roots, with most of his family's heritage being from Mississippi. He later joined the Sons of Confederate Veterans and regularly used the word "Confederate".