I'm a woman who stutters. Ironically enough, I get more or less the same vibes you seem to from men. As in, they'll approach me, making it clear they find me attractive or interesting or whatever, but as soon as I open my mouth they deflate and then they're gone in minutes. I'm a strong believer that people like that are not worthy of attention either way, but still... not great for one's self esteem. Beside this, I have to specify that I did get into a relationship and a few talking stages, and that they went kind of nowhere mostly because of my personality, not my stutter. I think the best course of action is to try and defy other people's expectations a bit: most assume, when they hear you stutter, that you're kind of meek and that you don't wan to talk about it. Therefore, talk about it, and flat out say that you have a speech disorder (casually, not like it's a life changing thing) and that you may need some extra time to let things out. Some won't really care and will leave anyway, but others will find you more interesting. Taking the initiative does miracles, even though it won't always work.