These people of Benjamin are not the soldiers who fled, but the ones who were still in the city.
"everyone who was in the city"
"along the way"
See in Reader