You are not restrained by us

"We have not restrained you" or "We have not given you any reason to stop loving us".

you are restrained in your own hearts

"your own hearts are restraining you" or "you have stopped loving us for your own reasons"