Are you guys like 14 years old? Hard to imagine a group of adults roasting their stuttering coworker. If I were you, I would prepare a very strong comeback. Wait till someone makes fun of you again in front of other people and then say something like: "I'm sorry that my neurological damage bothers you, it must be very annoying for you" Or something even better. But make sure it's in front of other people and make sure they really feel like the bad guy.