@allallalalalla You deserve better, and you need to have the hard conversation with him. Fear of breaking up is not a healthy reason to stay together; he has deceived and disrespected you.
I feel so much for what you're going through, the pain and angst must feel crushing at times... but if your relationship can't survive him the aftermath of him reading an honest letter about how you feel, that should tell you all you need to know about your future together 😞