We don't have to make the distinction "psychological" as opposed to "physical". Stutter is influenced by what we do, as a whole. It also varies from person to person what influencers there are, and which way they point. Besides, what you're describing is in the anecdotal. It might look different when presented via a properly configured study (double blind/control and all that). Take care that you don't self-medicate for the wrong reasons.