I don't believe in the "a clean job is a good job" idea. One of the best jobs I ever had left me completely filthy after each day, but I was way happier than I am now, in a much cleaner job. Just because a job is physically easy doesn't mean it is mentally easy and not draining. Ultimately you'll have to make this decision to leave or not, but it seems like you don't have much to look forward to in this job. And by that logic it would probably be pretty easy to find another job of a similar caliber if you chose to leave and regretted it. Also, I'd like to add that it doesn't matter if your managers were joking or not. Stuttering is not something you do intentionally. If they're going to have a problem with it, it's their problem not yours. Don't even waste the mental space on this aspect of it. (It does sound like they were joking though if it helps)