It's cause auditory feedback affects how fluent we are. When you speak, you're also hearing your own voice as well. I'm not read on the mechanism behind how auditory feedback gets incorporated in speech processing system of the human brain. But what I do know is the brain area (wernikes area) that's responsible for processing auditory feedback is under-activated during speech for people who stutter. So any background noise or other people speaking while you're talking is further limiting your ability to hear yourself speak, which leads to it being harder to talk / more difficult blocks.