Process users one at a time through the whole pipeline, added --queue [file] and --wait. Fixed self posts never being saved.

Users now go through profile, posts, content info and downloads one by one (preparing the next
user while the current one downloads), saving each index as soon as that user is done. The queue
is re-read between users, users are removed once archived, and a lock file lets a second
--queue invocation hand its users to the running process.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

CLAUDE.md

@@ -17,13 +17,24 @@ Node.js CLI that archives reddit users' posts (and direct links to supported med
 
 ## Architecture
 
-The pipeline runs from `src/app.js`:
+`src/app.js` processes users **one at a time through the whole pipeline**, via `runUsers()`. While one user's media downloads, the next user is already being prepared (lookahead of one). Each user's index is saved as soon as that user is done. Do not go back to processing stage by stage across all users: that meant hours of reddit requests before the first download, everything lost on a crash, and no way to queue new users.
 
-1. **Sources** (`src/sources/`): `getUserPosts` / `getPosts` fetch the user profile, social links and submissions through the reddit client, plus the existing index (`getIndex`, YAML at `library.index.file`).
+Per user:
+
+1. **Sources** (`src/sources/`): `getUserPosts(username, tracker)` resolves to `{ status, user }`. Status is `active`, `empty`, `suspended`, `deleted` or `error`. It fetches the profile (`user.js`), the existing index (`getIndex`, YAML at `library.index.file`), social links if needed, and submissions.
 2. **Curate** (`src/curate/`): `posts.js` normalizes reddit post objects, applies filters (index, dates, limit, ignore, include/exclude, duplicates) and runs `dissectLink` to choose a host method. `user.js` normalizes the profile.
-3. **Info** (`src/fetch/info.js`): calls `methods[host.method]` (in `src/methods/`) to resolve each post into `{ album, items[] }`. If that fails, it falls back to reddit preview images (`redditPreview`). Some methods export `{ fetchInfo, fetchPredata }`; predata (e.g. a RedGifs temporary token) is fetched once per run.
+3. **Info** (`src/fetch/info.js`): calls `methods[host.method]` (in `src/methods/`) to resolve each post into `{ album, items[] }`. If that fails, it falls back to reddit preview images (`redditPreview`). Some methods export `{ fetchInfo, fetchPredata }`. Predata (e.g. a RedGifs temporary token) is cached in `app.js` for `fetch.predataTtl`.
 4. **Content** (`src/fetch/content.js`): downloads items (`src/fetch/item.js`, using bhttp), dedupes by blake2 hash, saves to a path built by `src/interpolate.js` from `config.library` patterns, muxes video+audio, writes EXIF and finally updates the index (`src/save/writeToIndex.js`).
 
+Where users come from: `src/queue.js` provides `createListSource` (`--users`/`--file-users`) and `createQueueSource` (`--queue [file]`, default `queue`). `--posts` and `--fetch` still run as small batches.
+
+### Queue (`--queue [file]`, `--wait`)
+
+- The queue file is reread before each next user. A user is removed after finishing, including when suspended or deleted. Users with status `error` stay queued and are skipped for the rest of the run, so they can't loop.
+- `<queue>.lock` (holding a pid, created with `wx`) ensures only one process runs a queue. A second `--queue --users …` process only appends its users and exits. Without `--queue`, the lock is ignored.
+- At the end of a run, the lock is released first and the queue checked once more, which closes the race with a process that appended while it still saw the lock.
+- List files are edited with `src/utils/listFile.js`: reread right before writing, only the matching lines touched, replaced atomically via rename.
+
 ### Reddit client (`src/reddit/client.js`), the core of this branch
 
 - Calls reddit's public `.json` endpoints on `www.reddit.com` (`/user/x/about.json`, `/user/x/submitted.json`, `/comments/id.json`, `/api/morechildren.json`) with `raw_json=1`. There is no OAuth.
@@ -42,7 +53,7 @@ The pipeline runs from `src/app.js`:
 ## Account status and cleanup
 
 - `src/sources/user.js` `fetchUser()` classifies accounts as `active`, `suspended` (about.json returns 200 with `is_suspended: true`; there is no date), `deleted` (JSON 404, which may also be a shadowban) or `error`. **`error` (network failure, 5xx, HTML bot block) must never count as dead**; only genuine JSON API errors carry `error.status` (see `requestJson`).
-- Suspended and deleted users are skipped after that one request. `src/sources/accountStatus.js` records them in `users_dead.yml` (`config.users.dead`) with `firstSeen`. Once they have been dead for `retainDays[status]` (default 90), they are removed from the `--file-users` file, never from `--users`. A record is dropped if the account turns out active again. Files are only written at the end of a run.
+- Suspended and deleted users are skipped after that one request. `src/sources/accountStatus.js` records them in `users_dead.yml` (`config.users.dead`) with `firstSeen`. Once they have been dead for `retainDays[status]` (default 90), they are removed from the `--file-users` file, never from `--users`. A record is dropped if the account turns out active again. The record is saved after every user.
 - `--cleanup` runs only this check (one request per user) without archiving. Regular archive runs do it too unless `users.dead.duringArchive` is false.
 - Social links (an extra HTML request) are only fetched when the profile description will actually be saved, i.e. it isn't indexed yet or `--redownload-profile` is set.