So I went up ... and I turned back

The other men with Nehemiah also followed him. AT: "So we went up ... and we turned back" (See: rc://en/ta/man/translate/figs-synecdoche)

by the Valley Gate

"through the Valley Gate"