The opposite of passion might be indifference. The opposite of love, is hate/intolerance.

Overcoming internal fear of rejection
So, where does fear come from? I believe it is a learned behavior. Fear marks experience and an ability to grasp the concepts required for survival. We don't say that to walk around in a state of being afraid is good! But acknowledging fear leads to a dissection of the fear and gives you an opportunity to overcome it. Much the same as education/critical thinking relieves ignorance. Its all about resolution. Remember the saying "curiosity killed the cat". Well, I don't think it was curiosity. The little critter was fearless due to a wee bit of ignorance!

Live Life To Its Fullest Potential
Relationships are not responsible for "draining" either partner. It is up to the individual to live life to its fullest potential and not to depend on a partner or relationship to meet the needs that are best met individually. You or any third party do/does not break up a relationship. Each person is his own agent. If a woman chooses to leave her husband for another man, it is the woman's responsibility for making that decision. She could always say, "no". If a man leaves his wife for another woman, same deal. Men and women should strive to be complete, independent people Men and women should strive to be whole, complete, independent people before developing relationships. If they all did that there would be fewer divorces. Love and hate both require passion. It is some sort of feeling, that is why in many deep loving relationships the end is often stormy.