Well sometimes I feel even more anxious if I don't stutter, because I'm just waiting for it to happen. That anticipatory anxiety can be worse that the actual "right after the event" anxiety. I guess paradoxically we're just not used to talking freely and without an inner censor, and when it happens it's new and foreign to us and so it tires us.