mailcat/restart_prompt.md

Restart Prompt

You are continuing the mailcat project in /Users/hanswienen/Documents/Development/Vibes/mailcat.

Project Goal

Build a reliable mailbox cleanup and automation toolkit for australius.nl.

The project started as an offline analysis of a local mailbox export and is being moved toward an IMAP-based sorter because Sieve does not work with the email provider. The intended production model is:

• Email is hosted at australius.nl.
• A VPS is available at vps.australius.nl.
• The test mailbox is backup@australius.nl.
• The future automation should run on the VPS, preferably via SSH deployment and an IMAP IDLE daemon.
• The first end-to-end test should mirror production mail into backup@australius.nl, then sort only the backup mailbox.

Repository State

Git workflow:

• The repo has branches main, claude, and codex.
• Work should continue on codex unless the user says otherwise.
• Follow the user’s global workflow: initialize Git when needed, branch before work in existing repos, and commit after each user-prompted change using the required three-section commit message format.

Tracked files include Python scripts, generated reports, logs, caches, and project docs.

Ignored local data includes:

• mailbox/
• **/*.eml
• config.json
• __pycache__/
• bytecode files
• .DS_Store

Do not commit email .eml files or credentials.

Current Decisions

Sorting policy:

• INBOX.Facturen - verwerkt is the only folder treated as already sorted.
• Sent, Drafts, Trash, and Spam are excluded from sorting.
• Every other mailbox folder is treated as a source.
• Do not use Archief as a final destination.
• Existing Archief.* folders are source folders; route matching messages out of them into functional destination folders.
• If an archived/source message does not match a rule, leave it in place.

Destination policy:

• Final destination folders are functional folders such as Financieel, Werk, Diensten, Nieuwsbrieven, Bestellingen, Mobiliteit, and Technisch.
• Final destination folders under Archief should not be created.
• Technisch and Technisch.DMARC are valid destination folders.

Sieve:

• mailrules.sieve exists as historical/source material.
• Sieve deployment is not viable with the provider.
• Future work should move operational automation to Python over IMAP/SMTP.

Important Current Implementation Notes

verplaats_bestaand.py has already been changed so:

• MAP_RENAMES is intentionally empty.
• Source folders are selected by list_source_folders().
• Excluded source folders are Sent, Drafts, Trash, and Spam, with both bare and INBOX. names where relevant.
• ALREADY_SORTED_FOLDERS = {"INBOX.Facturen - verwerkt"}.
• Step 2 routes all eligible source folders, not just INBOX.
• Move success now requires both UID COPY and UID STORE +FLAGS \Deleted to return OK.
• Step 3 no longer treats INBOX.Facturen - verwerkt as a source.

maak_mappen.py has already been changed so:

• It no longer creates Archief.* destination folders.
• It creates Technisch and Technisch.DMARC.

Last verification performed:

• python3 -m py_compile *.py
• route/folder consistency check reported missing_targets 0
• archive target check reported archief_targets 0

Reports and Findings

The prior script review is in reports/script_review_findings.md.

Key earlier findings:

• The old redistribution script selected some source folders read-only and then tried to delete messages.
• The old logging could mark partial moves as complete.
• The old implementation only routed INBOX.
• Folder creation and route targets were inconsistent before recent fixes.

What To Do Next

Recommended next work:

1. Refactor shared IMAP utilities:

   • folder list parsing
   • mUTF-7 encoding/decoding
   • folder creation
   • UID fetch
   • safe copy/delete/expunge
   • Message-ID dedupe

2. Refactor shared routing logic:

   • extract the route table from verplaats_bestaand.py
   • support both historical sorting and live sorting from one source of truth
   • preserve invoice priority and quarter-folder routing

3. Harden scripts:

   • kopieer_naar_backup.py should not mark a folder complete if append failures occur
   • verplaats_bestaand.py should log failures with source folder, UID, destination, and server response
   • add a dry-run audit mode showing planned moves by source and destination

4. Add VPS automation:

   • create an IMAP IDLE daemon for backup@australius.nl
   • deploy via SSH to vps.australius.nl
   • use a non-root sudo user
   • install as a systemd service
   • keep secrets in an ignored config.json with restrictive permissions

5. Test on backup@australius.nl before production:

   • mirror production mail into backup
   • create target folders in backup
   • run dry-run sorter on backup
   • run actual sorter on backup
   • verify that only excluded folders and unmatched messages remain unsorted

Persistent File Rule

Before every future commit in this project, rewrite this restart_prompt.md file so it describes the current state at that commit. Do not append. Replace the content with a fresh, accurate reconstruction prompt.