I have the same experience. I don't stutter in public settings, but it increases exponentially in one-on-one settings. Sounds counterintuitive, but I think it might be because I don't feel as self-conscious in front of bigger audiences.