Here is one of the lines that I live by:
When in doubt, do without.
Not sure if someone came up with this line before I did, but it has served me well.
Whether it’s trying to decide whether to buy something at the store or whether to pursue someone or remain in a relationship... if you have doubts, there is usually a good reason. Maybe not always, but certainly more often than not.
I guess what I’m trying to say is, if you doubt your parter enough to want to access their phone, something is probably wrong with them (or wrong with you).