As long as youre married, having sex with someone who isnt your spouse counts as adultery. If youre separated at the time however, no one except your in-laws or the Pope is going to care.