I don't think it would look too awkward. I've met a few fluent people who mouthed words as another person was speaking; it just seemed to be a habit. This is great if it helps you.