the fact that you're this thoughtful about it probably already makes him more comfortable than you realize you don't have to bring up the stutter directly. next time there's a natural pause you could just say something like "hey no rush at all, i'm not going anywhere" and leave it at that. no big conversation needed, no acknowledgment of the stutter itself. just signal through your behavior that you're not in a hurry. people who stutter can usually tell pretty quickly who's patient and who isn't. sounds like he already knows you're one of the good ones