# My own presence will go

"I will go" 

# go with you ... give you

The word "you" here refers to Moses. It is singular. 

# I will give you rest

"I will let you rest"