id_tn_l3/Cleanup process.md

# Indonesian tN issues

The correct .md file format is documented at https://www.techadvancement.com/community/train/how-to-format-translation-notes-for-publishing-v-mast

## Cleanup steps:

1. Initial inspection of the id_tn_l3 data. Preliminary report to team on 11/12/19.
2. Conference call on 11/20/19. (Chuck, John, Craig, MAx, Tabitha, Christine)
3. Documented correct file format at  https://www.techadvancement.com/community/train/how-to-format-translation-notes-for-publishing-v-mast.
4. More extensive analysis and documentation of deviations.
5. Forked the Indonesian repository to https://wacs.bibletranslationtools.org/lversaw/id_tn_l3. This is the WACS workspace for data cleanup for this project.

## Automated cleanup steps:

1. Removed the first line of every note file other than the intro.md files. Also removed the second line if blank. 
    * Consistently, notes files other than intro.md had two extraneous lines at the top.
    * RISK: might possibly have deleted some valid data.
    * 23,055 files affected
2. Removed the first two lines from intro.md files that start with some variation of "# Pendahuluan"
    * 446 files affected
3. Removed lines containing empty HTML comments, and the lines following, if blank.
    * All HTML comment tags found were empty comments.
    * 469 files affected
4. Converted instances of & nbsp; to a single space.
    * 11694 files affected
5. Removed top line of file if blank. Consolidated consecutive blank lines elsewhere in file.
    * 842 files affected
6. Removed instances of <o:p></o:p> and <o:p> </o:p>
    * They had no apparent purpose or meaning.
    * 620 files affected
7. Removed blank headers and the blank line following (if any).
    * 1858 files affected
8. Fixed language code in tA links. Replaced rc://en/ with rc://id/
    * 20,065 files affected
9. Removed blank lines between list items.
    * 427 files affected
10. Removed high level hash tags in files showing the first classic pattern of corrupted heading levels.
    * Classic pattern means: First heading at level 1. Subsequent headings alternate higher level to level 1.
    * Ends with higher level heading. No untagged text lines anywhere.
    * 681 files affected
11. Promoted headings to level 1 in files showing the second classic pattern of corrupted heading levels.
    * Classic pattern means: First heading at level 2 or higher. Subsequent headings always the same level.
    * Plain text lines alternate with headings. Ends with plain text line.
    * 1665 files affected
12. Removed top two lines of files meeting these criteria:
    * At least 5 lines long
    * First line contains a verse reference (space followed by digits, colon, and digits)
    * Second line is blank, and third line starts with hash mark
    * 3394 files affected
    * RISK: might possibly have deleted some valid data.
13. Removed "Kata-kata Terjemahan" section from files that had it.
    * Those are from an older tN version.
    * 4817 files affected
14. Reapplied #11.
    * 475 files affected
15. Reapplied #10
    * 241 files affected
16. Volunteers manually edited the files identified in *issues.txt*.
    * about 4300 files affected
17. Renamed all folders and files in Psalms from 2-digit to 3-digit names.
    * 2385 files/folders affected

Remaining issues (will be) documented in *issues.txt*.

## Remaining steps:

TBD