Most people are pretty fluent talking to pets, babies, or when they're alone. I suspect you don't have much of a problem around your family because you feel comfortable stuttering around them. I personally still sometimes stutter talking to myself, but this only began a couple years ago. I know a few other severe stutterers who stutter alone sometimes. It's not common though.