The men of Israel had given ground to Benjamin, because they were trusting in the men ... outside Gibeah

From here until the end of verse 41 is background information, explaining how the ambush defeated the Benjamites.

had given ground to Benjamin

"had allowed Benjamin to move forward"