Sounds par for the course. You may also find that over time, the difficulty landscape (of speech items) will shift around.