mailcat/restart_prompt.md

# Restart Prompt

You are continuing the `mailcat` project in `/Users/hanswienen/Documents/Development/Vibes/mailcat`.

## Project Goal

Build a reliable mailbox cleanup and automation toolkit for `australius.nl`.

The project started as an offline analysis of a local mailbox export and is being moved toward an IMAP-based sorter because Sieve does not work with the email provider. The intended production model is:

- Email is hosted at `australius.nl`.
- A VPS is available at `vps.australius.nl`.
- The test mailbox is `backup@australius.nl`.
- The future automation should run on the VPS, preferably via SSH deployment and an IMAP IDLE daemon.
- The first end-to-end test should mirror production mail into `backup@australius.nl`, then sort only the backup mailbox.

## Repository State

Git workflow:

- The repo has branches `main`, `claude`, and `codex`.
- Work should continue on `codex` unless the user says otherwise.
- Follow the user’s global workflow: initialize Git when needed, branch before work in existing repos, and commit after each user-prompted change using the required three-section commit message format.

Tracked files include Python scripts, generated reports, logs, caches, and project docs.

Ignored local data includes:

- `mailbox/`
- `**/*.eml`
- `config.json`
- `__pycache__/`
- bytecode files
- `.DS_Store`

Do not commit email `.eml` files or credentials.

## Current Decisions

Sorting policy:

- `INBOX.Facturen - verwerkt` is the only folder treated as already sorted.
- `Sent`, `Drafts`, `Trash`, and `Spam` are excluded from sorting.
- Every other mailbox folder is treated as a source.
- Do not use `Archief` as a final destination.
- Existing `Archief.*` folders are source folders; route matching messages out of them into functional destination folders.
- If an archived/source message does not match a rule, leave it in place.

Destination policy:

- Final destination folders are functional folders such as `Financieel`, `Werk`, `Diensten`, `Nieuwsbrieven`, `Bestellingen`, `Mobiliteit`, and `Technisch`.
- Final destination folders under `Archief` should not be created.
- `Technisch` and `Technisch.DMARC` are valid destination folders.

Sieve:

- `mailrules.sieve` exists as historical/source material.
- Sieve deployment is not viable with the provider.
- Future work should move operational automation to Python over IMAP/SMTP.

## Important Current Implementation Notes

`verplaats_bestaand.py` has already been changed so:

- `MAP_RENAMES` is intentionally empty.
- Source folders are selected by `list_source_folders()`.
- Excluded source folders are `Sent`, `Drafts`, `Trash`, and `Spam`, with both bare and `INBOX.` names where relevant.
- `ALREADY_SORTED_FOLDERS = {"INBOX.Facturen - verwerkt"}`.
- Step 2 routes all eligible source folders, not just `INBOX`.
- Move success now requires both `UID COPY` and `UID STORE +FLAGS \Deleted` to return `OK`.
- Step 3 no longer treats `INBOX.Facturen - verwerkt` as a source.

`maak_mappen.py` has already been changed so:

- It no longer creates `Archief.*` destination folders.
- It creates `Technisch` and `Technisch.DMARC`.

Last verification performed:

- `python3 -m py_compile *.py`
- route/folder consistency check reported `missing_targets 0`
- archive target check reported `archief_targets 0`

## Reports and Findings

The prior script review is in `reports/script_review_findings.md`.

Key earlier findings:

- The old redistribution script selected some source folders read-only and then tried to delete messages.
- The old logging could mark partial moves as complete.
- The old implementation only routed `INBOX`.
- Folder creation and route targets were inconsistent before recent fixes.

## What To Do Next

Recommended next work:

1. Refactor shared IMAP utilities:
   - folder list parsing
   - mUTF-7 encoding/decoding
   - folder creation
   - UID fetch
   - safe copy/delete/expunge
   - Message-ID dedupe

2. Refactor shared routing logic:
   - extract the route table from `verplaats_bestaand.py`
   - support both historical sorting and live sorting from one source of truth
   - preserve invoice priority and quarter-folder routing

3. Harden scripts:
   - `kopieer_naar_backup.py` should not mark a folder complete if append failures occur
   - `verplaats_bestaand.py` should log failures with source folder, UID, destination, and server response
   - add a dry-run audit mode showing planned moves by source and destination

4. Add VPS automation:
   - create an IMAP IDLE daemon for `backup@australius.nl`
   - deploy via SSH to `vps.australius.nl`
   - use a non-root sudo user
   - install as a systemd service
   - keep secrets in an ignored `config.json` with restrictive permissions

5. Test on `backup@australius.nl` before production:
   - mirror production mail into backup
   - create target folders in backup
   - run dry-run sorter on backup
   - run actual sorter on backup
   - verify that only excluded folders and unmatched messages remain unsorted

## Persistent File Rule

Before every future commit in this project, rewrite this `restart_prompt.md` file so it describes the current state at that commit. Do not append. Replace the content with a fresh, accurate reconstruction prompt.