went down
This means that they traveled from a higher place to a lower place. Cana is southwest of Capernaum and is at a higher elevation.
his brothers
The word "brothers" includes both brothers and sisters. All Jesus' brothers and sisters were younger than he was.