Your colleagues didn’t hire you, the company did. What do you owe your colleagues? Don’t mind whatever they’re saying. People are always going to talk about you behind your back, stuttering or not, unfortunately it’s just the reality of some ‘toxic’ workplaces. As I have previous experience with a similar situation, my advice is just forget them. Keep your head down and focus on the work and job you’ve been blessed with, once your colleagues see that their petty talks behind your back are useless, they’ll eventually stop. If it’s to the point of harassment, I would contact management or HR department and explain the situation. They can’t take disciplinary action toward you if you’re bringing up a potentially unsafe work environment. Keep your head up. Fuck the haters, you got this.