As a person who stutters I would appreciate my server picking up on it and just being patient. Don't look bored or fidgety. He has a voice and is using it. By cutting him off or trying to expedite the process you are taking away his voice.