What you're doing is right. Most stutters I know dont want special treatment but basic respect, which is exactly what you're doing. Honestly, I wouldnt overthink it and try not to care. If the listener looks worried, that can make the stutterer more self conscious. There's a lot of good information in this thread too: http://www.reddit.com/r/Stutter/comments/1pzmdw/to_those_of_you_who_stutter_what_is_the_ideal_way/ Thanks for asking and trying to learn and being thoughtful!