That's the attitude to have. Next time you are struggling instead of hanging up just say something like "Please bear with me as I stutter and the phone can be a but difficult for me." I noticed something as simple as this can alleviate a lot of the pressure to speak perfectly.