I've usually found the opposite. I feel more comfortable when there's background noise and all attention isn't on my speech. Edit: However, very loud noise like a concert where I have to basically yell makes it hard for me to talk.