I've avoided hobbies in the past, I've taken up hobbies that I didn't really enjoy too. But I realized something only very recently. The only reason I feel I do not wish to partake in certain hobbies is because I'm trying to avoid stuttering. That doesn't work. I've started to take up hobbies recently that I enjoy and that envolve speaking, but I just say 'to hell with it' and do what I want. There's going to be 10x if not more patient and decent people than there will be dickheads. My advice, from personal experience, if someone insults you over it, laughs or whatever it may be - just ask them politely not to be so ignorant. 99.9999% of the time this will work if you're not hostile, but sincere about it. If not, either leave or stand up for yourself if you feel necessary. Hope this all helps!