Yuk's lack of eye contact

I don't understand why Sian and Minseol are so into him when he can't even look at them or even face their direction when they're talking.

He always seem bored or annoyed or constipated. I know his image is that of a "beast"/tough guy but isn't it so rude to not look at the person you are talking to?