The speaker talks about the difference in trust and openness between being a part of a small group like the hammer, and being in the SEAL teams where there is a need for more secrecy and tight-lipped communication due to better resources and higher stakes.