Sexual attraction

Sexual attraction. This subject is a murky and painful one for me. I have what I consider to be big problems in this area. Here's my biggest concern: the 2 men that I have been the most sexually attracted to in my life were also the 2 most toxic men that I ever had in my life. It's rather scary actually; for me, sexual attraction and toxic are positively correlated.
I need to break this as soon as possible but I am afraid and I don't know where to begin. Also, what if I never get the chance to experience that type of sexual chemistry with someone non-toxic? Maybe emotionally painful, brief, and meaningless sexual encounters with toxic men are the most I can hope for. God, I sure hope not!!!