Jesus based that statement on his knowledge of them, that they were from below, he was from above. They were of this world and he was not of this world.