Here "old yeast" refers to the sins that the people were committing. "Unleavened dough" is the type of bread that God told the Israelites was acceptable to eat when celebrating Passover. Paul is telling his audience to stop sinning so that they may be acceptable to God.