I think opening the conversation with a disclaimer might be a good idea. It's not like your stutter is going to stay a secret, they're going to find out eventually. Probably sooner than later. Opening the conversation by "coming clean" will establish you aren't ashamed about your impediment and you get it out in the open and move on. When I need to speak to a group, for a presentation or something, I'll sometimes open with a disclaimer about my speech. I'll say something like "I have a bit of a stutter and I apologize for any confusions it may cause". Personally, doing that helps me because once I do so I feel like I have nothing to hide anymore, which helps me relax which in turn helps fluency.