he probably senses the power he has over you and it makes him feel good about himself that he can click his finger and you come running. Then, when he's satisfied/come to his (twisted) senses, he pushes you away cos he can and for effect. I knew a bloke like that when I was a teenager who admitted that he liked to make a girl fall so completely for him and then play pull/push with her, toy with her feelings and the more he did, the more the girls loved him. Be strong and tell him to F off. Your boat has sailed, you're (happily) married and he's still waving on the quay! Sad loser! I bet he treated you badly when you were with him... not like physically but played with your mind and that.
That sounds so familiar it actually hurts.
I am happily married, I just can't stand the thought of people disliking me, it upsets me. xx
now forget about him. He's not worthy of your time, worry or thoughts... In fact he's not worthy of mine either! So, what have you got planned for the weekend? Hope your hubby is treating you to a nice meal out or drink out or something.