I feel the same way. My stuttering usually goes in cycles where I stutter all the time for 3 or so months to not stuttering for another 3 or so months and back. During the 'off' times I might stutter here or there but its less noticeable, even to myself. But when people say that, it makes me feel bad. I don't really know how to explain it. I guess kind of like my problem isn't a big deal even though it fills me with more anxiety and it makes me feel like shit inside. I know they don't mean it but that's how it feels to me.