Think of a stutter as a disability, which it is. Would you say the same things about a colleague in a wheelchair, who might need a bit more time for tasks due to limitations or mobility issues? If you can't bear to work with this person - who is disabled - then perhaps you should look for a new job.