You should be getting more sleep, consistently, every single night. Are you able to taper off the caffeine for a few weeks to see how your speech improves? Caffeine binds to the adenosine receptors in your brain, and also constricts blood vessels in your brain (resulting in less oxygen). When you block adenosine, you’re blocking a very important relaxant for your brain/body. When this happens, dopamine surges, and this is really bad for most stutterers.