While I understand what your saying, I don't think it's solely because of anxiety. You get people who develop a stutter after physical head trauma. There are cases of it running in the family. Then we have social influences like The Monster Study where people develop a stutter. There are just a lot of possible variables to consider.