Stuttering when speaking another language/stuttering on certain letters.

Stuttering when speaking another language/stuttering on certain letters. So, like most of you I imagine, I've stuttered since I started talking. Over the years, I've managed to really beat it down, and now, when speaking English, I am pretty decent at hiding it, most people don't even know I have a stutter until they've known me for a while. Every once in a while I'll get locked up but it's not a huge deal for the most part. (though, when telling a story or telling jokes it likes to screw with me from time to time.) Though, I've been learning Japanese as a matter of personal interest for about a year now, and I'm very mildly conversational... but It's the weirdest damn thing. Every time I speak it, ESPECIALLY in front of people, my stutter flares up like absolutely crazy. And is nearly IMPOSSIBLE to control. Like, all this time I've spent beating it down just completely goes out the window when I'm speaking in Japanese. It's such a downer. Especially when I'm trying to show off the little bit I actually know well, it always looks like I barely know what I'm talking about because I stammer so hard. On top of that, my stutter in Japanese remains mild on words that start with most letters... but, for whatever reason, M's and N's DESTROY me. They are literally massive roadblocks every time. I can start a Japanese sentence just fine, but as soon as I hit that M or N... my mild stutter turns into an excessive case, I just sit there going Nnnnnnnnnnnnnnnn....... nnnnnnnnnnnnnnnnnnnnnnn...... nnnnnnnnnnn.... it's literally the worst thing. Have any of you dealt with a stutter flaring up in a different language, and specifically a hard time getting out Ns and Ms? Can any share advice on either subject? I'd love to try to get to the bottom of this. Thanks for your time!