It might not be so much that you are negatively evaluating yourself. It could more be because of expectancy. You expect yourself to stutter so you do, even if not stressed.