For what it's worth, I think all teachers strive to be a source of comfort and security for their students. They want what's best for you and all students. You don't have to chastise him, but he can't hate you for sharing how you feel. I know if I were him, I would want to know if I accidentally did something to upset a student. I'd appreciate it. It is your choice and whatever you decide is perfectly alright :).