 DebaucheryLibrarian
|
18acb3a878
|
Added fetch concurrency.
|
2024-09-11 05:16:56 +02:00 |
|
DebaucheryLibrarian
|
91577a00b1
|
Added support for file with host IDs to ignore.
|
2024-09-11 05:16:56 +02:00 |
|
DebaucheryLibrarian
|
e087e47069
|
No longer redownload profile image and description when indexed, unless --redownload-profile is specified.
|
2024-09-11 05:16:56 +02:00 |
|
DebaucheryLibrarian
|
99c7d143f7
|
Fixed index file for single post fetching.
|
2024-09-11 05:16:56 +02:00 |
|
DebaucheryLibrarian
|
cdd5ed36bf
|
Added redownload option to ignore index file.
|
2024-09-11 05:16:56 +02:00 |
|
DebaucheryLibrarian
|
24a165cf48
|
Added date and indexed oldest/latest limit arguments.
|
2024-09-11 05:16:56 +02:00 |
|
DebaucheryLibrarian
|
86002ef00b
|
Accounting for indexed posts.
|
2024-09-11 05:16:56 +02:00 |
|
DebaucheryLibrarian
|
029351f228
|
Using YAML rather than TSV for index files. Improves both readability and reindexability.
|
2024-09-11 05:16:56 +02:00 |
|
DebaucheryLibrarian
|
f41b788183
|
Extracting indexed posts from fetched posts to allow for index rewrite flexibility.
|
2024-09-11 05:16:56 +02:00 |
|
DebaucheryLibrarian
|
c49e1edca0
|
Reading index file and ignoring already indexed content.
|
2024-09-11 05:16:56 +02:00 |
|
DebaucheryLibrarian
|
952392e0d9
|
Building user posts object after fetching user to ensure user fetched posts and directly fetched posts are added to the same user key. Refactor to make better use of functions. Moved profile detail saving call to content fetch. No longer attempting and failing to save profile details for deleted users (directory would not exist).
|
2024-09-11 05:16:56 +02:00 |
|
DebaucheryLibrarian
|
bba0f8f1d7
|
Fetching and curating posts per user without merge.
|
2024-09-11 05:16:56 +02:00 |
|
DebaucheryLibrarian
|
84836bc8c0
|
Refactored post retrieval so limit is applied per-user and ignores directly requested posts, and to start utilizing async/await.
|
2024-09-11 05:16:55 +02:00 |
|
DebaucheryLibrarian
|
c2cf26dde6
|
Refactored post fetching. Cleaner code that will now fetch archived posts even if the user no longer exists on reddit. Expanded gfycat URL detection.
|
2024-09-11 05:16:55 +02:00 |
|
DebaucheryLibrarian
|
e01979a757
|
Added include and exclude source arguments. Improved stream fetch failure handling and improved logging details.
|
2024-09-11 05:16:55 +02:00 |
|
DebaucheryLibrarian
|
60eeca09d8
|
Added support for m.imgur.com links. Changed curate ignore log message details.
|
2024-09-11 05:16:55 +02:00 |
|
DebaucheryLibrarian
|
1ae51b18f8
|
Added preview property. Removed debugging intercepts. Logging post permalink on failed info fetch.
|
2024-09-11 05:16:54 +02:00 |
|
DebaucheryLibrarian
|
f0e312f2b1
|
Added archive support, and the IP archive.
|
2024-09-11 05:16:54 +02:00 |
|
DebaucheryLibrarian
|
de64cc6d13
|
Added support for fetching individual posts. Improved use of yargs, --help now available. Refactored main app flow.
|
2024-09-11 05:16:54 +02:00 |
|
DebaucheryLibrarian
|
767ae082fa
|
Added ignore argument.
|
2024-09-11 05:16:54 +02:00 |
|
DebaucheryLibrarian
|
536c427140
|
Added support for muxing streams, specifically for reddit-hosted videos (now also supported).
|
2024-09-11 05:16:54 +02:00 |
|
DebaucheryLibrarian
|
84cb64e84c
|
Using metadata configuration rather than item description for EXIF save. Fixed dissect duplicate detection failing for empty host.
|
2024-09-11 05:16:54 +02:00 |
|
DebaucheryLibrarian
|
5a3d2dd030
|
Ignoring empty interpolated metadata. Added URL query wildcard to imgur and reddit images. Using host ID rather than URL to improve duplicate detection.
|
2024-09-11 05:16:54 +02:00 |
|
DebaucheryLibrarian
|
901029d9a1
|
Changed ignoreDuplicates to avoidDuplicates.
|
2024-09-11 05:16:53 +02:00 |
|
DebaucheryLibrarian
|
94e1575c30
|
Dealing with users without profile.
|
2024-09-11 05:16:53 +02:00 |
|
DebaucheryLibrarian
|
2b9801999b
|
Moved source into dedicated directory.
|
2024-09-11 05:16:53 +02:00 |