I would see this as an opportunity to sign up for a course (plenty out there that improve your speaking). You could see your job as exposure therapy, and opportunities to practice your speech. Your own private practice grounds. That's what I would do. Go for it. The only thing you got to lose is your current crappy job. One last advice: do not hide the fact that you stutter from your colleagues. Once you disclose this once, you'll feel way better. People appreciate when others open up. You can make friends quicker that way.