Possible meanings are 1) "you" refers to just Moses or 2) "you" refers to Moses and the leaders of Israel in future generations when they take a census.
The leaders only counted the males of Israel.
See in Reader