We are often very hard to ourselves and care way more about stuttering than all the other people. And the few people who do care don't matter. What will help you a lot is confidence. I never mentioned my stuttering at work, with new people etc until I was 31. A turning point was when I started to mention it and talk about it openly. People really don't care. How you get confidence is up to you, you can start small, changing your diet to something more healthy as example. Learning a new hobby, be creative ;)