# we who have believed enter that rest

"we who have believed will enter the place of rest" or "we who have believed will experience God's blessings of rest"
(Psalm 95:11)

# just as he said

"just as God said"

# As I swore in my wrath

"As I swore when I was very angry"

# They will never enter my rest

"They will never enter the place of rest" or "They will never experience my blessings of rest" 

# his works were finished

"he finished his works of creation" 

# from the foundation of the world

"at the beginning of the world" or "from the time he created the world". The author speaks of the world as if it were a building set on a foundation.