You are right.
Sex is not bad. Under the right conditions (within a marriage), sex is good, it brings new life and completes a marriage. But sex outside of marriage is not good. It often causes hurt relationship and emotional damage.
In an peer advisory group that I was in, we used to use this comparison:
Fire in itself is not bad. It gives us heat and light when in the fireplace; but a fire outside the fireplace is not good, it can even cause severe damage.