Trust is fundamental to a relationship. Do you trust him? Respect is also fundamental. Respect yourself. If someone thinks less of you for your speech impediment, that's their failing. Not yours. Sounds like you don't have to tell him, as he's already picked up on it? If that's the case, maybe you shouldn't consider disclosing your stutter, so much as disclosing your feelings about your stutter. Any way it goes, you'll be better for the experience. Best wishes.