Process users one at a time through the whole pipeline, added --queue [file] and --wait. Fixed self posts never being saved.
Users now go through profile, posts, content info and downloads one by one (preparing the next user while the current one downloads), saving each index as soon as that user is done. The queue is re-read between users, users are removed once archived, and a lock file lets a second --queue invocation hand its users to the running process. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -31,6 +31,14 @@ function getArgs() {
|
||||
describe: 'Load reddit usernames from file',
|
||||
type: 'string',
|
||||
})
|
||||
.option('queue', {
|
||||
describe: 'Work through a queue file of usernames (default: queue.file in config, \'queue\'), removing each user once archived and re-reading the file for new users as it goes. Users given with --users/--file-users are added to the queue. If another process is already working through the queue, only adds them and exits',
|
||||
type: 'string',
|
||||
})
|
||||
.option('wait', {
|
||||
describe: 'With --queue: keep running when the queue is empty, checking it for new users every queue.interval ms',
|
||||
type: 'boolean',
|
||||
})
|
||||
.option('cleanup', {
|
||||
describe: 'Only check the supplied users for suspended/deleted accounts: record them, and remove them from --file-users once dead for longer than configured. Does not archive anything',
|
||||
type: 'boolean',
|
||||
|
||||
Reference in New Issue
Block a user