I totally feel you. I used to stutter in my native language, but I learned to manage it and today I'm fluent. However, when it comes to english, I feel like a failure. The worst part is that I feel that I'm not taking advantage of my exchange program, since I tend to avoid situations where I have to speak in english and end up hanging only with people from my country.