Leaving a man alone

At what point do men just loose respect for you because you “keep going back”. Having a hard time not wanting to go back to this guy and I don’t usually have this issue. We can go months without talking then act like nothing happened. We both admit “we cant leave each other alone”, but it seems more he likes the attention and ego boost he gets.