It’s time to leave the party when you start to wonder if it’s time to leave the party.
When you start questioning if your relationship will last. Seriously, when you’re happy and in a good relationship, even a bad fight won’t really trigger those feelings (obviously this is highly dependent on your past — if you’ve been in an abusive relationship it might take a while before you trust that your SO won’t abuse you or leave you).
If you’re not comfortable with your SO. I think being comfortable around them is very important. This comes from someone with a lot of anxiety. –Sticky-Sticker