Interviewer said "You're acting like it's worse than it is"

Interviewer said "You're acting like it's worse than it is" So I had a phone interview and twice during the interview I apologized for my stutter and said, sorry if it's difficult to understand because of my stutter. Both times he said he can understand me perfectly but second time he said "You're acting like it's worse than it is". I don't know what to make of it, could it mean my stutter is not as bad as I think?