Extract shared mail routing and IMAP folder helpers
Prompt since previous commit: execute step 1 Centralized route targets, source-folder policy, and modified UTF-7 mailbox quoting. Verification: py_compile, route/folder consistency, and git diff --check passed.
This commit is contained in:
+51
-32
@@ -22,8 +22,6 @@ Git workflow:
|
||||
- Work should continue on `codex` unless the user says otherwise.
|
||||
- Follow the user’s global workflow: initialize Git when needed, branch before work in existing repos, and commit after each user-prompted change using the required three-section commit message format.
|
||||
|
||||
Tracked files include Python scripts, generated reports, logs, caches, and project docs.
|
||||
|
||||
Ignored local data includes:
|
||||
|
||||
- `mailbox/`
|
||||
@@ -58,28 +56,58 @@ Sieve:
|
||||
- Sieve deployment is not viable with the provider.
|
||||
- Future work should move operational automation to Python over IMAP/SMTP.
|
||||
|
||||
## Important Current Implementation Notes
|
||||
## Current Implementation Notes
|
||||
|
||||
`verplaats_bestaand.py` has already been changed so:
|
||||
Shared modules:
|
||||
|
||||
- `imap_utils.py` centralizes IMAP modified UTF-7 folder encoding/decoding, quoted mailbox names, LIST parsing, and folder listing.
|
||||
- `mail_routes.py` centralizes `PREFIX`, domain routes, invoice keywords, source-folder exclusions, already-sorted folders, invoice quarter routing, and generated destination folders.
|
||||
|
||||
Scripts using shared IMAP folder handling:
|
||||
|
||||
- `verplaats_bestaand.py`
|
||||
- `maak_mappen.py`
|
||||
- `kopieer_naar_backup.py`
|
||||
- `download_mailbox.py`
|
||||
- `dagelijks_overzicht.py`
|
||||
|
||||
`verplaats_bestaand.py` current behavior:
|
||||
|
||||
- `MAP_RENAMES` is intentionally empty.
|
||||
- Source folders are selected by `list_source_folders()`.
|
||||
- Excluded source folders are `Sent`, `Drafts`, `Trash`, and `Spam`, with both bare and `INBOX.` names where relevant.
|
||||
- `ALREADY_SORTED_FOLDERS = {"INBOX.Facturen - verwerkt"}`.
|
||||
- Source folders are selected by shared `list_folders()` plus `mail_routes.is_source_folder()`.
|
||||
- Step 2 routes all eligible source folders, not just `INBOX`.
|
||||
- Move success now requires both `UID COPY` and `UID STORE +FLAGS \Deleted` to return `OK`.
|
||||
- Move success requires both `UID COPY` and `UID STORE +FLAGS \Deleted` to return `OK`.
|
||||
- Step 3 no longer treats `INBOX.Facturen - verwerkt` as a source.
|
||||
- IMAP mailbox names are consistently quoted and encoded through `imap_utils.quote_mailbox()`.
|
||||
|
||||
`maak_mappen.py` has already been changed so:
|
||||
`maak_mappen.py` current behavior:
|
||||
|
||||
- Destination folders are generated from `mail_routes.destination_folders()`.
|
||||
- It no longer creates `Archief.*` destination folders.
|
||||
- It creates `Technisch` and `Technisch.DMARC`.
|
||||
- It creates all parent folders needed for route targets, including `Technisch` and `Technisch.DMARC`.
|
||||
- IMAP mailbox names are consistently quoted and encoded through `imap_utils.quote_mailbox()`.
|
||||
|
||||
Last verification performed:
|
||||
`kopieer_naar_backup.py` current behavior:
|
||||
|
||||
- Mirrors folders from one configured IMAP account to another.
|
||||
- Skips `INBOX.Trash` and `INBOX.Spam`.
|
||||
- Deduplicates by `Message-ID` within each destination folder.
|
||||
- Uses shared folder listing and mailbox quoting/encoding for select/create/append operations.
|
||||
|
||||
## Last Verification
|
||||
|
||||
The latest verification performed:
|
||||
|
||||
- `python3 -m py_compile *.py`
|
||||
- route/folder consistency check reported `missing_targets 0`
|
||||
- archive target check reported `archief_targets 0`
|
||||
- `python3 -c 'from mail_routes import DOMAIN_ROUTES,destination_folders; targets={target for _, target in DOMAIN_ROUTES}; folders=set(destination_folders()); print("routes", len(targets)); print("folders", len(folders)); print("missing_targets", sorted(targets-folders)); print("archief_targets", sorted(t for t in targets if t.startswith("Archief") or ".Archief" in t))'`
|
||||
- `git diff --check`
|
||||
|
||||
Expected consistency result:
|
||||
|
||||
- `routes 72`
|
||||
- `folders 127`
|
||||
- `missing_targets []`
|
||||
- `archief_targets []`
|
||||
|
||||
## Reports and Findings
|
||||
|
||||
@@ -96,32 +124,24 @@ Key earlier findings:
|
||||
|
||||
Recommended next work:
|
||||
|
||||
1. Refactor shared IMAP utilities:
|
||||
- folder list parsing
|
||||
- mUTF-7 encoding/decoding
|
||||
- folder creation
|
||||
- UID fetch
|
||||
- safe copy/delete/expunge
|
||||
- Message-ID dedupe
|
||||
1. Harden move and mirror auditability:
|
||||
- `kopieer_naar_backup.py` should not mark a folder complete if append failures occur.
|
||||
- `verplaats_bestaand.py` should log failures with source folder, UID, destination, and server response.
|
||||
- Add a dry-run audit mode showing planned moves by source and destination.
|
||||
|
||||
2. Refactor shared routing logic:
|
||||
- extract the route table from `verplaats_bestaand.py`
|
||||
- support both historical sorting and live sorting from one source of truth
|
||||
- preserve invoice priority and quarter-folder routing
|
||||
2. Add shared operation helpers where useful:
|
||||
- UID fetch wrappers.
|
||||
- safe copy/delete/expunge helper.
|
||||
- Message-ID dedupe helper reused by live automation.
|
||||
|
||||
3. Harden scripts:
|
||||
- `kopieer_naar_backup.py` should not mark a folder complete if append failures occur
|
||||
- `verplaats_bestaand.py` should log failures with source folder, UID, destination, and server response
|
||||
- add a dry-run audit mode showing planned moves by source and destination
|
||||
|
||||
4. Add VPS automation:
|
||||
3. Add VPS automation:
|
||||
- create an IMAP IDLE daemon for `backup@australius.nl`
|
||||
- deploy via SSH to `vps.australius.nl`
|
||||
- use a non-root sudo user
|
||||
- install as a systemd service
|
||||
- keep secrets in an ignored `config.json` with restrictive permissions
|
||||
|
||||
5. Test on `backup@australius.nl` before production:
|
||||
4. Test on `backup@australius.nl` before production:
|
||||
- mirror production mail into backup
|
||||
- create target folders in backup
|
||||
- run dry-run sorter on backup
|
||||
@@ -131,4 +151,3 @@ Recommended next work:
|
||||
## Persistent File Rule
|
||||
|
||||
Before every future commit in this project, rewrite this `restart_prompt.md` file so it describes the current state at that commit. Do not append. Replace the content with a fresh, accurate reconstruction prompt.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user